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Abstract ofWO0235848 

A data processing device for burying additional 
information in coded data created by encoding 
image data and correctly reproducing the image 
data and the additional data from the coded data 
without increasing the amount of coded data. A 
burying compression-encoder (11) encodes 
image data according to a predetermined 
encoding rule and destroys the encoding rule 
according to the additional information so as to 
bury the additional information. A decoder (12) 
restores burying coded data generated by 
burying the additional information in the coded 
data to the coded data encoded according to the 
encoding rule, and thereby the additional 
information is decoded and the coded data is 
decoded to image data. 
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(54) DATA PROCESSING DEVICE 

(57) The present invention relates to a data process- 
ing apparatus. Additional information is embedded in 
coded data obtained by encoding image data, without 
increasing the amount of the coded data, and the data 
in which the additional information is embedded is cor- 
rectly decoded into the image data and the additional 
information. An embedded compression encoder 11 en- 
codes image data according to a predetermined coding 



rule, and destroys the coding rule based on additional 
information, thereby embedding the additional informa- 
tion. A decoder 12 restores the embedded coded data, 
obtained by embedding the additional information in the 
coded data, into the coded data encoded according to 
the coding rule, thereby decoding the additional infor- 
mation and also decoding the coded data into the image 
data. 
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Description 




Technical Field 

[0001] The present invention relates to data process- 
ing apparatuses. More specifically, it relates to a data 
processing apparatus that allows information to be em- 
bedded in an image without degrading the quality of de- 
coded image or increasing the amount of data. 

Background Art 

[0002] Methods for embedding information in data 
without increasing the amount of the data exist, for ex- 
ample, changing the lowest bit or low-order two bits of 
digital audio data into information to be embedded. The 
method simply replaces lower bits of digital audio data 
with information to be embedded, taking advantage of 
the fact that the lower bits do not significantly affect the 
quality of sound. Thus, when the digital audio data is 
played back, the digital audio data with the information 
embedded therein is output as it is without restoring the 
lower bits. That is, since it is difficult to restore the lower 
bits, in which information is embedded, into the original 
digital audio data, and the lower bits do not significantly 
affect the quality of sound, the digital audio data is output 
with the information embedded therein. 
[0003] According to the method described above, 
however, data that differs from the original data is out- 
put. Thus, considerable effect is exerted upon the sound 
quality when the data is audio data or upon the picture 
quality when the data is video data. 

Disclosure of Invention 

[0004] The present invention has been made in view 
of the situation described above, and it allows informa- 
tion to be embedded, for example, in an image without 
degrading the picture quality or increasing the amount 
of data. 

[0005] A first data processing apparatus according to 
the present invention includes encoding means for en- 
coding first data to output coded data; and embedding 
means for embedding second data in the coded data 
output from the encoding means by modifying a portion 
of the coded data based on the second data. 
[0006] A first data processing method according to the 
present invention is such that first data is encoded to 
output coded data, and a portion of the coded data is 
modified based on the second data so that the second 
data is embedded in the coded data. 
[0007] A first storage medium according to the 
present invention stores such a program that first data 
is encoded to output coded data, and a portion of the 
coded data is modified based on the second data so that 
the second data is embedded in the coded data. 
[0008] According to the first data processing appara- 
tus, data processing method, and storage medium of the 
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present invention, fi^J^a is encoded to output coded 
data. Then, a portion of the coded data is modified 
based on the second data so that the second data is 
embedded in the coded data. 

5 [0009] A second data processing apparatus accord- 
ing to the present invention includes coding table crea- 
tion means for creating a coding table for encoding first 
data; modified coding table creation means for modify- 
ing the coding table created by the coding table creation 

10 means based on second data to create a modified cod- 
ing table; and embedded coded data generation means 
for encoding the first data based on the modified coding 
table to generate embedded coded data in which the 
second data is embedded. 

is [0010] A second data processing method according 
to the present invention is such that a coding table for 
encoding first data is created, the coding table created 
is modified based on second data to create a modified 
coding table, and the first data is encoded based on the 

20 modified coding table to generate embedded coded da- 
ta in which the second data is embedded. 
[0011] A second storage medium according to the 
present invention stores such a program that a coding 
table for encoding first data is created, the coding table 

25 created is modified based on second data to create a 
modified coding table, and the first data is encoded 
based on the modified coding table to generate embed- 
ded coded data in which the second data is embedded. 
[0012] According to the second data processing ap- 

30 paratus, data processing method, and storage medium 
of the present invention, a coding table for encoding first 
data is created, and the coding table created is modified 
based on second data to create a modified coding table. 
Then, the first data is encoded based on the modified 

35 coding table to generate embedded coded data in which 
the second data is embedded. 
[0013] A third data processing apparatus according to 
the present invention includes tentative decoding 
means for tentatively decoding embedded coded data 

40 encoded by embedding second data in first data, based 
on a coding table, to output tentative decoded data; ten- 
tative coding table creation means for creating a tenta- 
tive coding table based on the tentative decoded data; 
first decoded data obtaining means for decoding the em- 

4 $ bedded coded data based on the coding table and the 
tentative coding table to obtain first decoded data; and 
second decoded data obtaining means for obtaining 
second decoded data by comparing the coding table 
with the tentative coding table. 

so [0014] A third data processing method according to 
the present invention is such that embedded coded data 
encoded by embedding second data in first data is ten- 
tatively decoded based on a coding table to output ten- 
tative decoded data, a tentative coding table is created 

55 based on the tentative decoded data, the embedded 
coded data is decoded based on the coding table and 
the tentative coding table to obtain first decoded data, 
and the coding table is compared with the tentative cod- 
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ing table to obtain second dl^P&d data. 
[0015] A third storage medium according to the 
present invention stores such a program that embedded 
coded data encoded by embedding second data in first 
data is tentatively decoded based on a coding table to 
output tentative decoded data, a tentative coding table 
is created based on the tentative decoded data, the em- 
bedded coded data is decoded based on the coding ta- 
ble and the tentative coding table to obtain first decoded 
data, and the coding table is compared with the tentative 
coding table to obtain second decoded data. 
[001 6] According to the third data processing appara- 
tus, data processing method, and storage medium of the 
present invention, embedded coded data encoded by 
embedding second data in first data is tentatively de- 
coded based on a coding table to output tentative de- 
coded data. Furthermore, a tentative coding table is cre- 
ated based on the tentative decoded data, and the em- 
bedded coded data is decoded based on the coding ta- 
ble and the tentative coding table to obtain first decoded 
data. The coding table is compared with the tentative 
coding table to obtain second decoded data. 
[0017] A fourth data processing apparatus according 
to the present invention includes tentative decoding 
means for modifying a portion of embedded coded data 
encoded by embedding second data in first data accord- 
ing to an input parameter, and for tentatively decoding 
it based on a coding table to output tentative decoded 
data; re-encoding means for re-encoding the tentative 
decoded data to output re-encoded data; and decoding 
means for determining the parameter by comparing the 
embedded coded data and the re-encoded data, output- 
ting tentative decoded data as first decoded data, ob- 
tained by tentatively decoding, based on the coding ta- 
ble, the embedded coded data having been partially 
modified by the tentative decoding means based on the 
parameter, and for obtaining second decoded data cor- 
responding to the parameter. 

[0018] A fourth data processing method according to 
the present invention is such that a portion of embedded 
coded data encoded by embedding second data in first 
data is modified according to an input parameter, and 
tentatively decoded based on a coding table to output 
tentative decoded data, the tentative decoded data is 
re-encoded to output re-encoded data, and the param- 
eter is determined by comparing the embedded coded 
data and the re-encoded data, and tentative decoded 
data obtained by tentatively decoding, based on the 
coding table, the embedded coded data having been 
partially modified based on the parameter, is output as 
first decoded data, and for second decoded data corre- 
sponding to the parameter is also obtained. 
[0019] A fourth storage medium according to the 
present invention stores such a program that a portion 
of embedded coded data encoded by embedding sec- 
ond data in first data is modified according to an input 
parameter, and tentatively decoded based on a coding 
table to output tentative decoded data, the tentative de- 



coded data is re-en^Ed to output re-encoded data, 
and the parameter is determined by comparing the em- 
bedded coded data and the re-encoded data, and ten- 
tative decoded data obtained by tentatively decoding, 

5 based on the coding table, the embedded coded data 
having been partially modified based on the parameter, 
is output as first decoded data, and for second decoded 
data corresponding to the parameter is also obtained. 
[0020] According to the fourth data processing appa- 

10 ratus, data processing method, and storage medium of 
the present invention, a portion of embedded coded da- 
ta encoded by embedding second data in first data is 
modified according to an input parameter, and tentative- 
ly decoded based on a coding table to output tentative 

15 decoded data. Furthermore, the tentative decoded data 
is re-encoded to output re-encoded data. The parameter 
is determined by comparing the embedded coded data 
and the re-encoded data, and tentative decoded data 
obtained by tentatively decoding, based on the coding 

20 table, the embedded coded data having been partially 
modified based on the parameter, is output as first de- 
coded data; and for second decoded data correspond- 
ing to the parameter is also obtained. 
[0021] A fifth data processing apparatus according to 

25 the present invention includes encoding means for en- 
coding first data according to a coding rule to output cod- 
ed data; and modification means for modifying the cod- 
ing rule based on second data; wherein the encoding 
means encodes the first data according to a coding rule 

30 modified by the modification means, thereby generating 
embedded coded data in which the second data is em- 
bedded. 

[0022] Afifth data processing method according to the 
present invention is such that first data is encoded ac- 
35 cording to a coding rule to output coded data, the coding 
rule is modified based on second data, and the first data 
is encoded according to the modified coding rule to gen- 
erate embedded coded data in which the second data 
is embedded. 

40 [0023] A fifth storage medium according to the 
present invention stores such a program that first data 
is encoded according to a coding rule to output coded 
data, the coding rule is modified based on second data, 
and the first data is encoded according to the modified 

45 coding rule to generate embedded coded data in which 
the second data is embedded. 
[0024] According to the fifth data processing appara- 
tus, data processing method, and storage medium of the 
present invention, first data is encoded according to a 

50 coding rule to output coded data. Then, the coding rule 
is modified based on second data, and the first data is 
encoded according to the modified coding rule to gen- 
erate embedded coded data in which the second data 
is embedded. 

55 [0025] A sixth data processing apparatus according 
to the present invention includes first decoding means 
for decoding embedded coded data obtained by embed- 
ding second data in first data, to obtain coded data en- 
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coded according to an entropUPBing rule and to obtain 
the second data; and second decoding means for de- 
coding the coded data to obtain the first data. 
[0026] A sixth data processing method according to 
the present invention is such that embedded coded data 5 
obtained by embedding second data in first data is de- 
coded to obtain coded data encoded according to an 
entropy coding rule and to obtain the second data, and 
the coded data is decoded to obtain the first data. 
[0027] A sixth storage medium according to the 10 
present invention stores such a program that embedded 
coded data obtained by embedding second data in first 
data is decoded to obtain coded data encoded accord- 
ing to an entropy coding rule and to obtain the second 
data, and the coded data is decoded to obtain the first 1$ 
data. 

[0028] According to the sixth data processing appa- 
ratus, data processing method, and storage medium of 
the present invention, embedded coded data obtained 
by embedding second data in first data is decoded to 20 
obtain coded data encoded according to an entropy cod- 
ing rule and to obtain the second data. Furthermore, the 
coded data is decoded to obtain the first data. 
[0029] A data processing system according to the 
present invention includes an encoding apparatus in- 25 
eluding encoding means for encoding first data accord- 
ing to a coding rule to output coded data; and modifica- 
tion means for modifying the coding rule based on sec- 
ond data; wherein the encoding means encodes the first 
data according to a coding rule modified by the modifi- 30 
cation means to generate embedded data in which the 
second data is embedded, the data processing system 
also including a decoding apparatus including first de- 
coding means for decoding embedded coded data to 
obtain the coded data encoded according to the coding 35 
rule and to obtain the second data; and second decod- 
ing means for decoding the coded data to obtain the first 
data. 

[0030] According to the data processing system of the 
present invention, in the encoding apparatus, first data 40 
is encoded according to a coding rule to output coded 
data. Then, the coding rule is modified based on second 
data, and the first data is encoded according to the mod- 
ified coding rule to generate embedded data in which 
the second data is embedded. In the decoding appara- 45 
tus, embedded coded data is decoded to obtain the cod- 
ed data encoded according to the coding rule and to ob- 
tain the second data. Then, the coded data is decoded 
to obtain the first data. 

50 
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[0031] 



Fig. 1 is a diagram showing an example construc- 
tion of an embedded compression/decoding system 
according to an embodiment of the present inven- 
tion. 
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Fig. 2 is a blocl^BKjram showing a first example 
construction of an embedded compression encoder 
11. 

Fig. 3 is a block diagram showing an example con- 
struction of a variable-length encoding unit 23. 
Fig. 4 is a diagram showing variable-length decod- 
ing. 

Fig. 5 is a diagram showing the relationship be- 
tween the frequencies of occurrence of pixel values 
and code lengths of codes assigned to the pixel val- 
ues. 

Fig. 6 is a diagram showing an example Huffman 
table. 

Fig. 7 is a diagram showing codes assigned in ac- 
cordance with the frequencies of occurrence of pix- 
el values, and translation of the codes. 
Fig. 8 is a diagram showing an example of embed- 
ded coded data. 

Fig. 9 is a diagram showing a method for restoring 
embedded coded data into original coded data. 
Fig. 10 is a flowchart showing an embedded coding 
process. 

Fig. 11 is a flowchart showing a translation table cre- 
ation process. 

Fig. 12 is a flowchart showing a coded data trans- 
lation process. 

Fig. 1 3 is a block diagram showing a second exam- 
ple construction of the embedded compression en- 
coder 11. 

Fig. 14 is a block diagram showing a first example 
construction of a decoder 12. 
Fig. 1 5 is a block diagram showing an example con- 
struction of a variable-length decoding unit 52 and 
a variable-length decoding unit 56. 
Fig. 1 6 is a block diagram showing an example con- 
struction of a Huffman table creation unit 53. 
Fig. 1 7 is a block diagram showing an example con- 
struction of a reverse translation table creation unit 
54. 

Fig. 1 8 is a flowchart showing a decoding process. 
Fig. 1 9 is a block diagram showing a third example 
construction of the embedded compression encod- 
er 11. 

Fig. 20 is a flowchart showing an embedded coding 
process. 

Fig. 21 is a block diagram showing an example con- 
struction of an encoding unit 91 . 
Fig. 22 is a diagram showing encoding by vector 
quantization. 

Fig. 23 is a block diagram showing an example con- 
struction of an embedding unit 92. 
Fig. 24 is a diagram showing embedding of addi- 
tional information by line rotation. 
Fig. 25 is a flowchart showing an embedded coding 
process. 

Fig. 26 is a block diagram showing a second exam- 
ple construction of the decoder 1 2. 
Fig. 27 is a flowchart showing a decoding process. 
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Fig. 28 is a block diagramming an example con- 
struction of a coding rule restoring unit 121. 
Fig. 29 is a block diagram showing an example con- 
struction of a decoding unit 122. 
Fig. 30 is a block diagram showing an example con- 
struction of a determination unit 123. 
Fig. 31 is a diagram showing a process executed 
by the determination unit 123. 
Fig. 32 is a flowchart showing a decoding process. 
Fig. 33 is a block diagram showing a third example 
construction of the decoder 12. 
Fig. 34 is a block diagram showing an example con- 
struction of a computer according to an embodi- 
ment of the present invention. 

Best Mode for Carrying Out the Invention 

[0032] Fig. 1 is an example construction of an embed- 
ded compression/decoding system (a system refers to 
a logical combination of a plurality of apparatuses, irre- 
spective of whetherthe constituent apparatuses resides 
in the same casing) according to an embodiment of the 
present invention. 

[0033] The embedded compression/decoding system 
includes an encoding apparatus 1 and a decoding ap- 
paratus 2. The encoding apparatus 1 encodes, for ex- 
ample, an image to be encoded, and the decoding ap- 
paratus 2 decodes the result of the encoding into the 
original image. 

[0034] The encoding apparatus 1 includes an embed- 
ded compression encoder 11. To the embedded com- 
pression encoder 11, an image to be encoded and in- 
formation to be embedded in the image (hereinafter re- 
ferred to as additional information when appropriate) are 
supplied. The embedded compression encoder 11 en- 
codes the image (digital image data) for compression 
according to a predetermined coding rule, and modifies 
or destroys the coding rule based on the additional in- 
formation (digital data), thereby embedding the addi- 
tional information to yield embedded coded data for out- 
put. The embedded coded data output from the embed- 
ded compression encoder 11 is recorded on a recording 
medium 3, for example, a semiconductor memory, a 
magneto-optical disk, a magnetic disk, an optical disk, 
a magnetic tape, or a phase-change disk, or transmitted 
via a transmission medium 4, for example, a terrestrial 
wave, a satellite link, a CATV (Cable Television) net- 
work, the Internet, or a public network, and is thus pro- 
vided to the decoding apparatus 2. 
[0035] The decoding apparatus 2 includes the decod- 
er 12, which receives the embedded coded data provid- 
ed via the recording medium 3 or the transmission me- 
dium 4. The decoder 12 restores the embedded coded 
data into the coded data encoded according to the pre- 
determined coding rule, thereby decoding the additional 
information embedded therein and also decoding the 
coded data into the original image. The decoded image 
is supplied to and displayed on a monitor, not shown, or 
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the like. 




[0036] It is to be understood that the additional infor- 
mation may include, for example, text data or sound da- 
ta associated with the original image, a reduced version 

5 of the image, etc., and may include data irrelevant to the 
original image. That is, arbitrary data (including a pro- 
gram) can be used as the additional information. 
[0037] Furthermore, part of an image supplied to the 
embedded compression encoder 11 for encoding may 

10 also be used as the additional information. That is, it is 
possible to supply part of the image as additional infor- 
mation to the embedded compression encoder 11 while 
also supplying the rest of the image as data to be en- 
coded. 

15 [0038] Fig. 2 shows an example construction of the 
embedded compression encoder 11 shown in Fig. 1 . 
[0039] A frame memory 21 stores image data sup- 
plied to the embedded compression encoder 1 1 , for ex- 
ample, on a frame basis. An additional information 

20 memory 22 stores additional information supplied to the 
embedded compression encoder 11 . 
[0040] A variable-length encoding unit 23 applies var- 
iable-length coding or entropy coding, for example, Huff- 
man coding, on the image data stored in the frame mem- 

25 ory 2 1 , and supplies the resulti ng coded data to a coding 
rule destroying unit 25. The variable-length encoding 
unit 23 creates a Huffman table in the process of Huff- 
man encoding in a manner to be described later, and 
supplies the Huffman table to a translation table creation 

30 unit 24. Furthermore, the variable-length encoding unit 
23 supplies information related to Huffman table, to a 
MUX (multiplexer) 26, which will be required in obtaining 
the Huffman table output to the translation table creation 
unit 24. 

35 [0041] The translation table creation unit 24 creates, 
based on the additional information stored in the addi- 
tional information memory 22, a translation table for 
translating codes in the Huffman table supplied from the 
variable-length encoding unit 23. That is, the Huffman 

40 table defines association between values to be Huff- 
man-encoded (pixel values of the image herein) and 
codes having different code lengths (coded data), and 
the translation table creation unit 24 creates a transla- 
tion table for translating the codes in the Huffman table 

45 into codes based on the additional information. The 
translation table created by the translation table creation 
unit 24 is supplied to the coding rule destroying unit 25. 
[0042] The coding rule destroying unit 25 modifies or 
destroys the coding rule in the variable-length encoding 

50 unit 23 based on the additional information, thereby em- 
bedding the additional information. That is, the coding 
rule destroying unit 25 translates (manipulates) the cod- 
ed data (codes) output from the variable-length encod- 
ing unit 23, according to the translation table created by 

55 the translation table creation unit 24 based on the addi- 
tional information, thereby yielding coded data encoded 
according to a coding rule in which the coding rule in the 
variable-length encoding unit 23 is destroyed. The cod- 
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ed data encoded according tlj^destroyed coding rule 
is supplied from the coding rule destroying unit 25 to the 
MUX 26 as embedded coded data obtained by embed- 
ding additional information in the original coded data. 
[0043] The MUX 26 multiplexes the embedded coded 
data from the coding rule destroying unit 25 and the in- 
formation related to Huffman table from the variable- 
length encoding unit 23, outputting the multiplexed data. 
As described with reference to Fig. 1, the multiplexed 
data is supplied to the decoding apparatus 2 via the re- 
cording medium 3 or the transmission medium 4. 
[0044] Fig. 3 shows an example construction of the 
variable-length encoding unit 23 shown in Fig. 2. 
[0045] Each frame of the image data stored in the 
frame memory 21 shown in Fig. 2 is sequentially con- 
sidered, for example, in time order, and the image data 
of the frame under consideration is read. The image da- 
ta of the frame under consideration is supplied to a fre- 
quency table creation unit 31 and to an encoding unit 34. 
[0046] The frequency table creation unit 31 creates a 
frequency table for the pixels constituting the frame un- 
der consideration supplied thereto, in which each pixel 
value is associated with the frequency of occurrence 
thereof, and supplies it to a Huffman tree creation unit 
32, The frequency table is also supplied from the fre- 
quency table creation unit 31 to the MUX 26 shown in 
Fig. 3 as information related to Huffman table. 
[0047] Although the frequency table is used as infor- 
mation related to Huffman table in the embodiment 
shown in Fig. 3, the information related to Huffman table 
is not specifically limited as long as the information al- 
lows a Huffman table created by a Huffman table crea- 
tion unit 33, which will be described later, to be obtained. 
Thus, the information related to Huffman table may be, 
for example, the Huffman table itself created by the Huff- 
man table creation unit 33, as welt as the frequency ta- 
ble. 

[0048] The Huffman tree creation unit 32 creates what 
is called a Huffman tree based on the frequency table 
supplied from the frequency table creation unit 31 , and 
supplies it to the Huffman table creation unit 33. The 
Huffman table creation unit 33 creates a Huffman table 
based on the Huffman tree supplied from the Huffman 
tree creation unit 32. That is, the Huffman table creation 
unit 33 creates a Huffman table in which pixel values in 
the frame under consideration are each assigned a code 
having a shorter code length in accordance with the fre- 
quency of occurrence of the pixel value being higher (a 
code having a longer code length in accordance with the 
frequency of occurrence being lower). The Huffman ta- 
ble is supplied to the encoding unit 34 and also to the 
translation table creation unit 24 shown in Fig. 2. 
[0049] The encoding unit 34 sequentially considers 
each pixel in the frame under consideration supplied 
thereto, for example, in order of raster scanning, and 
translates the pixel value of the pixel under considera- 
tion into the corresponding code in the Huffman table 
supplied from the Huffman table creation unit 33, out- 
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putting it as coded 




[0050] In the variable-length encoding unit 23 con- 
structed as described above, the frequency table crea- 
tion unit 31 creates a frequency table for a frame under 

5 consideration, which is supplied to the Huffman tree cre- 
ation unit 32. The Huffman tree creation unit 32 creates 
a Huffman tree based on the frequency table supplied 
from the frequency table creation unit 31 , which is sup- 
plied to the Huffman table creation unit 33. The Huffman 

10 table creation unit 33 creates a Huffman table based on 
the Huffman tree supplied from the Huffman tree crea- 
tion unit 32, which is supplied to the encoding unit 34. 
The encoding unit 34 translates each pixel value in the 
frame under consideration into a code associated with 

15 the pixel value in the Huffman table, which is output as 
coded data. 

[0051] Now, a variable-length encoding process in the 
variable-length encoding unit 23 will be further de- 
scribed with reference to Fig. 4. 

20 [0052] Let is be supposed, for example, that a fre- 
quency table as shown in Fig. 4(A) has been created in 
the frequency table creation unit 31 . Fig. 4(A) shows pix- 
el values [0], [1], [2], [3], and [4] as having occurred five 
times, fourtimes, three times, twice, and once in a frame 

25 under consideration, respectively. 

[0053] In relation to the frequency table in Fig. 4(A), 
the Huffman tree creation unit 32 creates a Huffman tree 
in a bottom-up manner based on the frequency of oc- 
currence of each pixel value, for example, as shown in 

30 Figs. 4(B) to 4(E). 

[0054] More specifically, the Huffman tree creation 
unit 32 selects pixel values with the two lowest frequen- 
cies of occurrence from the pixel values in the frequency 
table, thereby forming a node. Furthermore, the Huff- 

35 man tree creation unit 32 assigns either bit "0" or bit "1 " 
to the one with the lower frequency of occurrence of the 
selected two pixel values, for example, "0", assigning 
"1" to the other. The Huffman tree creation unit 32 as- 
signs the sum of the frequencies of occurrence of the 

40 selected two pixel values to the node formed by the se- 
lected two pixel values as the frequency of occurrence 
of the node. 

[0055] Thus, in the frequency table shown in Fig. 4 
(A), pixel value [4] having a frequency of occurrence 1 

45 and pixel value [3] having a frequency of occurrence 2 
are selected, forming a node #1 , and 3 (= 1 + 2) is as- 
signed as the frequency of occurrence of the node #1 , 
as shown in Fig. 4(B). Furthermore, of the selected two 
pixel values [4] and [3], bit "0" is assigned to pixel value 

50 [4], having a lower frequency of occurrence, and bit "1 " 
is assigned to pixel value [3], having a higher frequency 
of occurrence. 

[0056] In Figs. 4(B) to 4(E), numerals representing 
frequencies of occurrence are shown in parentheses. 
55 [0057] If the frequencies of occurrence of the two se- 
lected pixel values are the same, bits "O" or "1" may be 
assigned to either one of the pixel values. However, to 
which of the pixel values bits M 0" or'T 1 is to be assigned 



6 



I 




EP 1 331 820 A1 



must be predetermined, forVZ^hple, assigning bit "0" 
to the smaller pixel value. 

[0058] The Huffman tree creation unit 32 repeats the 
same process until convergence to a single node Is 
achieved. 

[0059] Thus, from the state shown in Fig. 4(B), the 
node #1 and pixel value [2] each having a frequency of 
occurrence 3 are selected, so that the node #1 and pixel 
value [2] form a node #2, as shown in Fig. 4(C). Further- 
more, the frequency of occurrence of the node #2 is set 
to 6 (= 3 + 3), and bits "0" and "1" are assigned to the 
node #1 and pixel value [2], respectively. 
[0060] From the state shown in Fig. 4(C), pixel value 
[1] having a frequency of occurrence 4 and pixel value 
[0] having a frequency of occurrence 5 are selected, so 
that pixel values [1 ] and [0] form a node #3. Furthermore, 
the frequency of occurrence of the node #3 is set to 9 
(=4+5), and bits "0" and "1 " are assigned to pixel values 
[1] and [0], respectively, as shown in Fig. 4(D). 
[0061 ] From the state shown in Fig . 4(D) , the node #2 
having a frequency of occurrence 6 and the node #3 
having a frequency of occurrence 9 are selected, so that 
the nodes #2 and #3 form a node #4. Furthermore, the 
frequency of occurrence of the node #4 is set to 1 5 (= 6 
+ 9), and bits "0" and "1" are assigned to the nodes #2 
and #3, respectively, as shown in Fig. 4(E). 
[0062] In the state shown in Fig. 4(E), the nodes have 
been converged to the single node #4, so that the Huff- 
man tree is complete. The Huffman tree creation unit 32 
supplies the Huffman tree to the Huffman table creation 
unit 33. 

[0063] The Huffman table creation unit 33 recognizes 
a code assigned to each pixel value by tracking the Huff- 
man tree from the convergence node in the direction of 
a pixel value. 

[0064] More specifically, for example, the Huffman 
tree shown in Fig. 4(E) is tracked from the node #4 in 
the direction of pixel value [0], and a sequence of bits 
assigned to the nodes (or pixel values) is found as 
«0"-> ,, o ,, -> ,, 0". Accordingly, the Huffman table creation 
unit 33 recognizes that code "000" is assigned to pixel 
value [0]. Also, the Huffman tree shown in Fig. 4(E) is 
tracked from the node #4 in the direction of pixel value 
[1], and a sequence of bits assigned to the nodes is 
found as "0"-> , '0"-y , r. Accordingly, the Huffman table 
creation unit 33 recognizes that code "001" is assigned 
to pixel value [1]. 

[0065] Similarly, the Huffman table creation unit 33 
recognizes a code assigned to each pixel value, thereby 
creating a Huffman table representing association be- 
tween pixel values and codes. Thus, from the Huffman 
tree shown In Fig. 4(E), a Huffman table as shown in 
Fig. 4(F) is created. 

[0066] The variable-length encoding unit 23 may al- 
ternatively perform variable-length encoding, for exam- 
ple, in the manner disclosed in United States Patent No. 
5,021,782. 

[0067] In the Huffman table shown in Fig. 4(E), codes 
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'11", "10", "01", "Otf^hd "000" are assigned to pixel 
values [0], [1], [2], [3], and [4] having frequencies of oc- 
currence 5, 4, 3, 2, and 1 , respectively. Thus, basically, 
codes of shorter code lengths are assigned in accord- 
5 ance with the frequencies of occurrence being higher. 
[0068] In the Huffman table shown in Fig. 4(E), al- 
though pixel values [0], [1], [2], [3], and [4] have different 
frequencies of occurrence, codes in two bits are as- 
signed to pixel values [0], [1], and [2] having relatively 
10 higher frequencies of occurrence, whereas codes in 
three bits are assigned to pixel values [3] and [4] having 
relatively lower frequencies of occurrence. 
[0069] As described above, in a Huffman table, codes 
having the same code length are sometimes assigned 
is to pixel values having different frequencies of occur- 
rence. The relationship between frequencies of occur- 
rence of pixel values and length of codes assigned to 
the pixel values are generally as shown in Fig. 5. 
[0070] Referring to Fig. 5, assuming that n-bit codes 
20 are assigned to x pixel values (x is an integer not ex- 
ceeding 2 n ), the number of patterns of assignment of 
the n-bit codes to the x pixel values is X! (! denotes fac- 
torial), only one of which is adopted based on the rule 
for creating the Huffman tree described above. 
25 [0071] Even if the pattern of assignment of the n-bit 
codes to the x pixel value is modified, the amount of 
codes does not increase. That is, in a Huffman table, 
even if an n-bit code is assigned to a pixel value instead 
of another n-bit code, the code length assigned remains 
30 as n bits, and thus the amount of codes does not in- 
crease. 

[0072] Furthermore, even if the pattern of assignment 
of the n-bit codes to the x pixel values is modified, the 
pattern of code assignment can be restored, for exam- 
35 pie, based on the frequencies of occurrence of x pixel 
values. 

[0073] From the above, even if the pattern of assign- 
ment of codes of the same length to pixel values is mod- 
ified in a Huffman table, that is, even if a coding rule for 

^o variable-length encoding is destroyed, the amount of 
codes does not increase, and the modified assignment 
pattern can be restored to the original one. 
[0074] This implies that, by modifying the pattern of 
assignment of codes to pixel values based on some in- 

45 formation, information can be embedded without in- 
creasing the total amount of data while allowing the em- 
bedded information to be decoded without overhead. 
[0075] Fig. 6 shows an example of a Huffman table 
created for pixel values represented in eight bits (0 to 

50 255). In Fig. 6, in addition to pixel values and codes (cod- 
ed data) assigned to pixel values, the frequency of oc- 
currence of each of the pixel values is shown. 
[0076] Referring to Fig. 6, for example, with regard to 
seven pixel values from [12] to [18], a nine-bit code is 

55 assigned to each of the seven pixel values, and thus 71 
patterns of assignment of the nine-bit codes to the seven 
pixel values exist. Thus, by modifying the pattern of as- 
signment of the nine-bit codes to the seven pixel values, 
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int[log 2 7!] bits of information embedded, intn rep- 
resents the largest integer value not exceeding the val- 
ue in []. 

[0077] Fig. 7(A) is a graph showing the respective fre- 
quencies of occurrence of pixel values [1 2] to [1 8] shown 
in Fig. 6, and Fig, 7(B) shows nine-bit codes respectively 
assigned to pixel values [12] to [18] in Fig. 6. 
[0078] Let it be supposed that the code assignment 
shown in Fig. 7(B) is modified based on additional infor- 
mation within int[log 2 7!] bits, for example, as shown in 
Fig. 7(C). 

[0079] More specifically, in Fig. 7(B), codes 
"110111111", "110111010", "11 01 00001 "110011001", 
"11011000", "011101011", and "010001010" are as- 
signed to pixel values [12] to [18], respectively. In Fig. 7 
(C), the assignment is modified so that code 
"110111111", which has been assigned to pixel value 
[12], is now assigned to pixel value [15], code 
"110011001", which has been assigned to pixel value 
[15], is now assigned to pixel value [12], code 
"11011000", which has been assigned to pixel value 
[16], is now assigned to pixel value [17], code 
"011101011", which has been assigned to pixel value 
[17], is now assigned to pixel value [18], and code 
"010001010", which has been assigned to pixel value 
[1 8], is now assigned to pixel value [1 6]. Assignment of 
codes to other pixel values is not modified in Fig. 7. 
[0080] Also for pixel values to which codes of other 
code lengths are assigned, pattern of code assignment 
can be modified based on the additional information. 
Fig. 8 shows an example of embedded coded data, con- 
stituting coded data in which the pattern of code assign- 
ment shown in Fig. 6 has been modified based on the 
additional information. Fig. 8 shows, together with the 
embedded coded data, decoded pixel values obtained 
by variable-length decoding the embedded coded data 
according to the Huffman table shown in Fig. 6 (pixel 
values obtained by decoding the embedded coded data 
according to the Huffman table used in obtaining the 
coded data). 

[0081] Referring to Fig. 8, for example, with regard to 
the seven pixel values from [1 2] to [1 8], if the embedded 
coded data is decoded according to the Huffman table 
shown in Fig. 6, decoded pixel values as shown in Fig. 
9 are obtained. That is, letting pixel values obtained by 
decoding the embedded coded data according to the 
Huffman table used at the time of encoding be referred 
to as embedded decoded pixel value, codes 
"110011001", "110111010", "110100001", "110111111", 
"010001010", "11011000", and "011101011" are varia- 
ble-length decoded into embedded decoded pixel val- 
ues [1 5], [1 3], [1 4], [1 2], [1 8], [1 6], and [17], respectively. 
[0082] Fig. 9(B) shows counts of the frequencies of 
occurrence of the embedded decoded pixel values [12] 
to [1 8], Since codes assigned to pixel values [1 2] to [1 8] 
in the Huffman table shown in Fig. 6 are modified as 
shown in Figs. 7(B) and 7(C), the frequencies of occur- 
rence of pixel values [12], [15], and [16] to [18] do not 
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coincide with those j n Fig. 7(A), and more spe- 

cifically, they coincide with the frequencies of occur- 
rence of pixel values [15], [12], [18], [16], and [17] prior 
to variable-length encoding, respectively. 

5 [0083] However, since the frequencies of occurrence 
of original pixel values [12] to [18] (prior to variable- 
length encoding) and the frequencies of occurrence of 
pixel values [12] to [18] obtained by variable-length de- 
coding are supposed to respectively coincide with each 

10 other, it is unnatural (strange) if the frequencies of oc- 
currence differ between prior to variable-length encod- 
ing and after variable-length decoding. 
[0084] Accordingly, the pattern of assignment of 
codes (variable-length codes), having been modified 

is based on the additional information, can be restored to 
the original pattern by comparing the frequencies of oc- 
currence of the original pixel values shown in Fig. 7(A) 
with the frequencies of occurrence of the embedded de- 
coded pixel values shown in Fig. 9(B) and by detecting 

20 matching frequencies of occurrence. That is, the em- 
bedded coded data can be restored into the coded data 
encoded according to the Huffman table. Furthermore, 
the embedded additional information can be decoded 
based on how the pattern of code assignment is modi- 

25 fjed when the embedded coded data is restored into the 
coded data, and also, the original pixel values can be 
obtained by variable-length decoding the restored cod- 
ed data. 

[0085] More specifically, by comparing the frequen- 
ce cies of occurrence of the original pixel values shown in 
Fig. 7(A) with the frequencies of occurrence of the em- 
bedded decoded pixel values shown in Fig. 9(B), thef re- 
quencies of occurrence of the embedded decoded pixel 
values [15], [13], [14], [12], [17], [18], and [16] can be 
35 detected as coinciding with the frequencies of occur- 
rence of the original pixel values [12] to [18], respective- 
ly. 

[0086] Thus, embedded coded data "110011001", 
"110111010", "110100001", "110111111", "010001010", 

40 "11011000", and "011101011", from which embedded 
decoded pixel values [15], [13], [14], [12], [17], [18], and 
[1 6] have been obtained, can be determined as having 
been codes "110111111", "110111010", "110100001", 
"110011001", "11011000", "011101011", and 

45 "010001010" assigned to the original pixel values [12] 
to [1 8] in the Huffman table shown in Fig. 6 prior to em- 
bedding of the additional information. 
[0087] Thus, by translating the embedded coded data 
so that the frequencies of occurrence of the original pixel 

50 values shown in Fig. 7(A) coincide with the frequencies 
of occurrence of the embedded decoded pixel values 
shown in Fig. 9(B), as shown in Fig. 9(C), the coded data 
encoded according to the Huffman table shown in Fig. 
6 can be restored, and the additional information can be 

55 decoded based on the association between the embed- 
ded coded data and the restored coded data. Further- 
more, by variable-length decoding the restored coded 
data, the original pixel values can be obtained, as shown 
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in Fig. 9(D). 
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[0088J The embedded compression encoder 11 
shown in Fig. 2 performs an embedded coding process 
that does not increase the amount of data and that al- 
lows decoding without overhead, as described above. 
[0089] Now, an embedded coding process executed 
by the embedded compression encoder 1 1 shown in Fig. 
2 will be further described with reference to a flowchart 
shown in Fig. 10. 

[0090] In step S1, the variable-length encoding unit 
23 variable-length encodes the pixel value of each pixel 
in a frame under consideration supplied from the frame 
memory 21, outputting the resulting coded data to the 
coding rule destroying unit 25. Furthermore, in the var- 
iable-length encoding in step S1 , the variable-length en- 
coding unit 23 (Fig. 3) outputs a frequency table created 
by the frequency table creation unit 31 to the MUX 26 
as information related to Huffman table, and also out- 
puts a Huffman table created by the Huffman table cre- 
ation unit 33 to the translation table creation unit 24. 
[0091] Then, in step S2, the translation table creation 
unit 24 modifies the pattern of assignment of codes hav- 
ing the same code length in Huffman table supplied from 
the variable-length encoding unit 23, based on the ad- 
ditional information supplied from the additional infor- 
mation memory 22, and executes a translation table cre- 
ation process for creating atranslation table associating 
the codes prior to the modification with codes after the 
modification. The translation table is supplied from the 
translation table creation unit 24 to the coding rule de- 
stroying unit 25. 

[0092] In step S3, the coding rule destroying unit 25 
executes a coded data translation process for translat- 
ing the coded data supplied from the variable-length en- 
coding unit 23 into embedded coded data according to 
the translation table supplied from the translation table 
creation unit 24, supplying the resulting embedded cod- 
ed data to the MUX 26. 

[0093] In step S4, the MUX 26 multiplexes and out- 
puts the embedded coded data of the frame under con- 
sideration, supplied from the coding rule destroying unit 
25, and the information related to Huffman table, sup- 
plied from the variable-length encoding unit 23. The 
process then proceeds to step S5. 
[0094] In step S5, it is determined whether a frame 
next to the frame under consideration is stored in the 
frame memory 21 . If it is determined in step S5 that a 
next frame is stored in the frame memory 21, with the 
next frame a new frame to be considered, the process 
returns to step S1 , repeating the same process. 
[0095] If it is determined in step S5 that a next frame 
is not stored in the frame memory 21 , the embedded 
coding process is exited. 

[0096] Now, the translation table creation process ex- 
ecuted in step S2 shown in Fig. 10 by the translation 
table creation unit 24 shown in Fig. 2 will be further de- 
scribed with reference to a flowchart shown in Fig. 11 . 
[0097] In the translation table creation process, initial- 
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ly, in step S11 , the trarlMron table creation unit 24 con- 
siders a particular code length of codes in the Huffman 
table supplied from the variable-length encoding unit 23, 
recognizing the number of codes having the code length 
5 under consideration. That is, the translation table crea- 
tion unit 24 recognizes the number of codes having the 
same code length as the code length under considera- 
tion. 

[0098] In step S1 2, the translation table creation unit 

10 24calculatesthenumberof bits of additional information 
that can be embedded in the codes having the code 
length under consideration, based on the number of 
codes having the code length under consideration, rec- 
ognized in step S11 . That is, letting the number of codes 

15 having the code length under consideration be denoted 
by x, the translation table creation unit 24 obtains the 
number of bits y of additional information that can be 
embedded, by calculating y = lnt[log 2 (X!)]. 
[0099] The process then proceeds to step S13, in 

20 which the translation table creation unit 24 reads the ad- 
ditional information for the number of bits y, calculated 
in step S12, from the additional information memory 22. 
The process then proceeds to step S14. In step S14, 
the translation table creation unit 24 creates a transla- 

25 tion table for the codes having the code length under 
consideration, based on the additional information read 
from the additional information memory 22 in step S13. 
That is, the translation table creation unit 24 modifies 
the pattern of assignment of x codes having the code 

30 length under consideration to pixel values, based on the 
y-bit additional information, thereby creating a transla- 
tion table for the codes having the code length under 
consideration, in which the codes prior to the modifica- 
tion are associated with codes after the modification. 

35 [0100] Then, in step S15, the translation table crea- 
tion unit 24 determines whether further additional infor- 
mation is stored in the additional information memory 
22. If it is determined that none is stored, the translation 
table creation unit 24 outputs translation tables created 

40 so far for the frame under consideration to the coding 
rule destroying unit 25 (Fig. 2). The translation table cre- 
ation process is then exited. 

[0101] If it is determined in step S15 that further ad- 
ditional information is stored in the additional informa- 

45 tion memory 22, the process proceeds to step S16, in 
which the translation table creation unit 24 determines 
whether a translation table has been created for codes 
of each code length in the Huffman table. If it is deter- 
mined that the creation is not complete, the process re- 

50 turns to step S11 . In this case, in step S11 , one of the 
code lengths for which atranslation table has not been 
created is selected as a new code length to be consid- 
ered, and then the same process is repeated. 
[01 02] If it is determined in step S1 6 that a translation 

55 table has been created for codes of each code length in 
the Huffman table, the translation table creation process 
is exited. 

[0103] Next, the coded data translation process exe- 
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cuted in step S3 shown in Fig^Gy the coding rule de- 
stroying unit 25 shown in Fig. 2 will be further described 
with reference to a flowchart shown in Fig. 12. 
[0104] As described above, the variable-length en- 
coding unit 23 considers each pixel in the frame under 5 
consideration in order of raster scanning, and variable- 
length encodes the pixel value of the pixel under con- 
sideration, sequentially outputting the resulting coded 
data. Letting the coded data output for the pixel under 
consideration by the variable-length encoding unit 23 be io 
referred to as coded data under consideration, initially, 
in step S21 , the coding rule destroying unit 25 recogniz- 
es the code length of the coded data under considera- 
tion. The process then proceeds to step 522. 
[0105] In step S22, the coding rule destroying unit 25 is 
translates the coded data under consideration accord- 
ing to the translation table associated with the code 
length of the coded data under consideration, recog- 
nized in step S21 , and outputs the result to the MUX 26 
(Fig. 2) as embedded coded data. 20 
[0106] The process then proceeds to step S23, in 
which the coding rule destroying unit 25 determines 
whether next coded data supplied from the variable- 
length encoding unit 23 exits. If it is determined that next 
coded data exists, the coding rule destroying unit 25 se- 25 
lects the next coded data as new coded data to be con- 
sidered. The process then returnstostep S21 , repeating 
the same process. 

[0107] If it is determined in step S23 that next coded 
data does not exist, the coded data translation process 30 
is exited. 

[0108] The embedded compression encoder 11 
shown in Fig. 2 yields embedded coded data, in which 
additional information is embedded, by translating, ac- 
cording to a translation table, coded data yielded by var- 35 
iable-length encoding in the variable-length encoding 
unit 23. Alternatively, for example, the variable-length 
encoding unit 23 may create a Huffman table in which 
the association of pixel values with codes is modified 
(hereinafter referred to as modified Huffman table when 40 
appropriate) and use the modified Huffman table to si- 
multaneously perform variable-length encoding and 
embedding of additional information. That is, the varia- 
ble-length encoding unit 23 may perform variable-length 
encoding based on a destroyed coding rule. 45 
[01 09] Fig. 1 3 shows an example construction of the 
embedded compression encoder 11 that executes such 
an embedded coding process. In the figure, parts cor- 
responding to those shown in Fig. 2 or Fig. 3 are desig- 
nated by the same numerals, and descriptions thereof so 
will hereinafter be omitted as appropriate. The embed- 
ded compression encoder 11 shown in Fig. 13 does not 
include the coding rule destroying unit 25; instead, a 
coding rule destroying unit 41 is included in the variable- 
length encoding unit 23. The embedded compression 55 
encoder 1 1 is otherwise constructed the same as shown 
in Fig. 2. 

[0110] To the coding rule destroying unit 41 , a Huff- 
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man table output fronWTe Huffman table creation unit 
33 and a translation table output from the translation ta- 
ble creation unit 24 are supplied. 
[0111] The coding rule destroying unit 41 modifies 
codes (coded data) associated with pixel values in the 
Huffman table into embedded coded data associated 
with the codes in the translation table, thereby modifying 
the Huffman table supplied from the Huffman table cre- 
ation unit 33 into a modified Huffman table in which the 
pixel values are associated with the embedded coded 
data. The modified Huffman table is supplied to the en- 
coding unit 34, which translates the pixel values accord- 
ing to the modified Huffman table. 
[0112] Thus, in Fig. 13, the encoding unit 34 outputs 
the embedded coded data. 

[0113] Fig. 14 shows an example construction of the 
decoder 1 2 shown in Fig. 1 , in a case where the embed- 
ded compression encoder 11 is constructed as shown 
in Fig. 2 or Fig. 13. 

[0114] A DEMUX (demultiplexer) 51 demultiplexes, 
on a frame basis, data provided from the embedded 
compression encoder 11 shown in Fig. 2 or Fig. 13 into 
embedded coded data and information related to Huff- 
man table, supplying the embedded coded data for each 
frame to a variable-length decoding unit 52 and to a cod- 
ing rule restoring unit 55 while supplying the information 
related to Huffman table to the variable-length decoding 
unit 52, a reverse translation table creation unit 54, and 
to a variable-length decoding unit 56. 
[0115] The variable-length decoding unit 52 sequen- 
tially considers each frame of the embedded coded data 
and the information related to Huffman table output from 
the DEMUX 51 , and creates a Huffman table based on 
a frequency table constituting the information related to 
Huffman table associated with the frame under consid- 
eration, in the same manner as in the variable-length 
encoding unit 23 shown in Fig. 3, supplying it to the re- 
verse translation table creation unit 54. 
[01 16] The Huffman table created from the frequency 
table constituting the information related to Huffman ta- 
ble is the same one as that used for variable-length en- 
coding in the variable-length encoding unit 23, and will 
hereinafter be referred to as a true Huffman table when 
appropriate. The frequency table constituting the infor- 
mation related to Huffman table, used to obtain the true 
Huffman table, will hereinafter be referred to as a true 
frequency table when appropriate. 
[0117] The variable-length decoding unit 52 then var- 
iable-length decodes the embedded coded data of the 
frame under consideration according to the true Huff- 
man table, and supplies the resulting decoded pixel val- 
ues, that is, embedded decoded pixel values, to the 
Huffman table creation unit 53. 
[0118] The Huffman table creation unit 53 creates a 
Huffman table for variable-length encoding the embed- 
ded decoded pixel values of the frame under consider- 
ation, supplied from the variable-length decoding unit 
52, and supplies the Huffman table and the frequency 
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table created in the process ofWfating the Huffman ta- 
ble to the reverse translation table creation unit 54. 
[01 19] The Huffman table created by the Huffman ta- 
ble creation unit 53 is used for variable-length encoding 
the embedded decoded pixel values (obtained by vari- s 
able-length decoding the embedded coded data accord- 
ing to the true Huffman table), and basically does not 
allow correct decoding of coded data. Thus, the Huff- 
man table obtained from the embedded decoded pixel 
values will hereinafter be referred to as a false Huffman 10 
table when appropriate, as opposed to the true Huffman 
table. The frequency table used for obtaining the false 
Huffman table will hereinafter be referred to as a false 
frequency table when appropriate, as opposed to the 
true frequency table. 15 
[0120] The reverse translation table creation unit 54 
creates a reverse translation table for translating the 
embedded coded data into the original coded data, 
based on the true Huffman table supplied from the var- 
iable-length decoding unit 52, the true frequency table 20 
constituting the information related to Huffman table, 
supplied from the DEMUX 51 , and the false Huffman 
table and the false frequency table supplied from the 
Huffman table creation unit 53. That is, the reverse 
translation table creation unit 54 creates the same re- 25 
verse translation table as the translation table created 
by the translation table creation unit 24 shown in Fig. 2 
(or Fig. 13). The reverse translation table is supplied to 
the coding rule restoring unit 55. 

[0121] The coding rule restoring unit 55 restores (de- 30 
codes) the embedded coded data supplied from the DE- 
MUX 51 into the coded data encoded according to the 
true Huffman table constituting the coding rule, accord- 
ing to the reverse translation table supplied from the re- 
verse translation table creation unit 54. Furthermore, the 55 
coding rule restoring unit 55 decodes the additional in- 
formation embedded in the embedded coded data, 
based on the association between the embedded coded 
data and the.restored coded data, that is, the reverse 
translation table. The coding rule restoring unit 55 out- *o 
puts the restored coded data to the variable-length de- 
coding unit 56 and also outputs the additional informa- 
tion that has been decoded as decoded additional infor- 
mation. 

[0122] The variable-length decoding unit 56 creates *s 
a true Huffman table from the information related to Huff- 
man table supplied from the DEMUX 51 , and variable- 
length decodes the coded data supplied from the coding 
rule restoring unit 55 based on the true Huffman table, 
outputting the resulting decoded pixel values. so 
[0123] Fig. 1 5 shows an example construction of the 
variable-length decoding unit 52 shown in Fig. 14. 
[0124] The embedded coded data output from the 
DEMUX 51 is supplied to a decoding unit 63, and the 
frequency table constituting the information related to 55 
Huffman table, output from the DEMUX 51, is supplied 
to a Huffman tree creation unit 61 . 
[0125] The Huffman tree creation unit 61 creates a 
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Huffman tree from th^Wquency table associated with 
the frame under consideration, and supplies it to a Huff- 
man table creation unit 62, similarly to the Huffman tree 
creation unit 32 shown in Fig. 3. 
[0126] The Huffman table creation unit 62 creates a 
Huffman table based on the Huffman tree supplied from 
the Huffman tree creation unit 61 , and supplies it to the 
decoding unit 63, similarly to the Huffman table creation 
unit 33 shown in Fig. 3. 

[0127] The decoding unit 63 decodes the embedded 
coded data supplied thereto into pixel values (embed- 
ded decoded pixel values) according to the Huffman ta- 
ble supplied from the Huffman table creation unit 62, and 
outputs the result. 

[0128] The variable-length decoding unit 56 shown in 
Fig. 14 is constructed the same as the variable-length 
decoding unit 52 shown in Fig. 15. 
[0129] Fig. 16 shows an example construction of the 
Huffman table creation unit 53 shown in Fig, 14. 
[0130] In the Huffman table creation unit 53, the em- 
bedded decoded pixel values output from the variable- 
length decoding unit 52 are suppliedto a frequency table 

71 . The frequency table 71 , a Huffman tree creation unit 

72, and a Huffman table creation unit 73 executes sim- 
ilar processes as the frequency table creation unit 31 , 
the Huffman tree creation unit 32, and the Huffman table 
creation unit 33 shown in Fig. 3, respectively. 

[0131] Thus, the frequency table creation unit 71 cre- 
ates a false frequency table, that is, a frequency table 
for the embedded decoded pixel values of the frame un- 
der consideration (not a frequency table for the original 
pixel values), which is supplied to the reverse translation 
table creation unit 54 (Fig. 14). The Huffman table cre- 
ation unit 73 creates a false Huffman table, that is, a 
Huffman table for translating the embedded decoded 
pixel values of the frame under consideration into codes 
having code lengths in accordance with the frequencies 
of occurrence thereof (not a Huffman table for the orig- 
inal pixel values), which is supplied to the reverse trans- 
lation table creation unit 54. 

[0132] Fig. 1 7 shows an example construction of the 
reverse translation table creation unit 54 shown in Fig. 
14. 

[0133] As described above, to the reverse translation 
table creation unit 54, the true Huffman table, the true 
frequency table, the false Huffman table, and the false 
frequency table are supplied. Based on these tables, the 
reverse translation table creation unit 54 recognizes the 
association between the embedded coded data and 
coded data encoded according to the true Huffman ta- 
ble, and creates a reverse translation table defining the 
association, as described with reference to Figs. 6 to 9. 
[0134] The true Huffman table and the false Huffman 
table are supplied to a code association unit 82, while 
the true frequency table and the false frequency table 
are supplied to a comparison unit 81 . 
[0135] The comparison unit 81 compares the true fre- 
quency table with the false frequency table to detect, for 
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each code length, pixel valuesWfrie same frequency of 
occurrence in the true frequency table and the false fre- 
quency table. Furthermore, the comparison unit 81 cre- 
ates a pixel value association table in which the pixel 
values of the same frequency in the true frequency table 
and the false frequency table are associated with each 
other, and supplies it to the code association unit 82. 
[01 36] The code association unit 82 searches the true 
Huffman table and the false Huffman table for the pixel 
values associated with each other in the pixel value as- 
sociation table supplied from the comparison unit 81, 
and associates codes respectively associated with the 
pixel values with each other, thereby creating a reverse 
translation table. The reverse translation table is sup- 
plied to the coding rule restoring unit 55. 
[01 37] Now, a decoding process executed by the de- 
coder 12 shown in Fig. 14 will be described with refer- 
ence to a flowchart shown in Fig. 18. 
[0138] The DEMUX 51 demultiplexes data supplied 
thereto into embedded coded data and information re- 
lated to Huffman table, supplying the embedded coded 
data for each frame to the variable-length decoding unit 
52 and to the coding rule restoring unit 55 while supply- 
ing the information related to Huffman table to the vari- 
able-length decoding unit 52, the reverse translation ta- 
ble creation unit 54, and to the variable-length decoding 
unit 56. 

[0139] In step S31 , the variable-length decoding unit 
52 sequentially considers each frame of the embedded 
coded data and the information related to Huffman table 
output from the DEMUX 51, and variable-length de- 
codes the embedded coded data of the frame under 
consideration. 

[0140] More specifically, the variable-length decoding 
unit 52 creates a Huffman table (true Huffman table) 
from a frequency table (true frequency table) constitut- 
ing the information related to Huffman table of the frame 
underconsideration, and supplies ittothe reverse trans- 
lation table creation unit 54. Furthermore, the variable- 
length decoding unit 52 variable-length decodes the em- 
bedded coded data of theframe underconsideration ac- 
cording to the true Huffman table, and supplies the re- 
sulting embedded decoded pixel values to the Huffman 
table creation unit 53. 

[0141] The process then proceeds to step S32, in 
which the Huffman table creation unit 53 creates a fre- 
quency table (false frequency table) from the embedded 
decoded pixel values of the frame under consideration, 
supplied from the variable-length decoding unit 52, and 
further creates a false Huffman table from the false fre- 
quency table. The false frequency table and the false 
Huffman table are supplied to the reverse translation ta- 
ble creation unit 54. 

[0142] In step S33, the reverse translation table cre- 
ation unit 54 creates a reverse translation table based 
on the true Huffman table, the true frequency table con- 
stituting the information related to Huffman table, the 
false Huffman table, and the false frequency table, as 
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described with refer§R5£to Fig. 17, and supplies it to 
the coding rule restoring unit 55. 
[0143] In step S34, the coding rule restoring unit 55 
decodes the additional information embedded in the 

5 embedded coded data, by referencing the association 
between the embedded coded data and coded data of 
each code length in the reverse translation table sup- 
plied from the reverse translation table creation unit 54, 
and outputs the resulting decoded additional informa- 

10 tion. 

[0144] Furthermore, in step S35, the coding rule re- 
storing unit 55 translates the embedded coded data sup- 
plied from the DEMUX 51 into coded data by referencing 
the reverse translation table, and supplies it to the var- 

15 iable-length decoding unit 56. 

[0145] The variable-length decoding unit 56 creates 
a true H uffm an table from the inf ormatio n related to H uff- 
man table supplied from the DEMUX 51 , and variable- 
length decodes the coded data supplied from the coding 

20 rule restoring unit 55 based on the true Huffman table, 
outputting the resulting decoded pixel values. 
[0146] The process then proceeds to step S37, in 
which it is determined whether embedded coded data 
and information related to Huffman table for a next frame 

25 has been output from the DEMUX 51 . If it is determined 
that they have been output, with the next frame a new 
frame to be considered, the process returns to step S31 , 
repeating the same process. 

[0147] If it is determined in step S37 that embedded 
30 coded data and information related to Huffman table for 
a next frame have not been output from the DEMUX 51 , 
the decoding process is exited. 
[0148] As described earlier, instead of the frequency 
table, the Huffman table itself may be used as the infor- 
ms mation related to Huffman table. In that case, however, 
the Huffman table constituting the information related to 
Huffman table must be created such that the frequen- 
cies of occurrence of pixel values to which codes (coded 
data) of the same length are assigned can be recog- 
40 nized as being higher or lower relative to each other by 
referencing the Huffman table (according to a rule that 
allows such recognition). 

[0149] Furthermore, although Huffman tables are cre- 
ated on a frame basis in the example described above, 

45 alternatively, Huffman tables may be created, for exam- 
ple, on a field basis, or on a basis of two or more frames. 
[0150] Furthermore, Huffman tables may be created 
in advance, for example, using a predetermined image 
data, so that the embedded compression encoder 11 

50 and the decoder 1 2 execute processes according to the 
Huffman tables created in advance. In that case, how- 
ever, if Huffman tables created from image data to be 
processed do not coincide with the Huffman tables cre- 
ated from the predetermined image data, the decoder 

55 1 2 will not be able to restore embedded coded data into 
coded data. Thus, additional information to be embed- 
ded must be limited only to such image data from which 
the same Huffman tables as those created from the pre- 
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determined image data can beTJBtained. 
[0151] Fig. 19 shows another example construction of 
the embedded compression encoder 1 1 shown in Fig. 1 . 
[0152] To an encoding unit 91 , image data to be en- 
coded is supplied. The encoding unit 91 encodes the 5 
image data according to a predetermined coding rule, 
outputting the resulting coded data to an embedding unit 
92. 

[0153] To the embedding unit 92, additional informa- 
tion is supplied. The embedding unit 92 manipulates the 10 
coded data supplied from the encoding unit 91 based 
on the additional information to obtain coded data en- 
coded according to a destroyed coding rule, thereby em- 
bedding the additional information in the coded data. 
[0154] Now, a process executed by the embedded « 
compression encoder 11 shown in Fig. 19 (embedded 
coding process) will be described with reference to a 
flowchart shown in Fig. 20. 

[01 55] To the encoding unit 91 , image data to be en- 
coded is supplied, for example, on a frame basis. The 20 
encoding unit 91 sequentially considers each frame, 
and in step S41 , it encodes image data of the frame un- 
der consideration according to a predetermined coding 
rule. The encoding unit 91 then outputs the resulting 
coded data to the embedding unit 92. 25 
[01 56] In step S42, the embedding unit 92 modifies or 
destroys the coding rule in the encoding unit 91 based 
on the additional information, thereby embedding the 
additional information. That is, the embedding unit 92 
manipulates the coded data supplied from the encoding 30 
unit 91 based on the additional information, thereby em- 
bedding the additional information in the coded data. 
Furthermore, the embedding unit outputs embedded 
coded data obtained by embedding the additional infor- 
mation in the coded data. The process then proceeds 35 
to step S43. 

[0157] In step S43, the encoding unit 91 determines 
whether a next frame to be encoded exists. If it is deter- 
mined that a next frame exists, with the next frame as a 
new frame to be considered, the process returns to step 40 
S41 , repeating the same process. 
[0158] If it is determined in step S43 that no further 
frame exists to be encoded next, the embedded coding 
process is exited. 

[0159] Fig. 21 shows an example construction of the 45 
encoding unit 91 shown in Fig. 19. 
[0160] In the encoding unit 91 shown in Fig. 21, for 
example, if the pixel values constituting image data are 
represented in RGB (Red, Green, and Blue), each of the 
pixel values is vector-quantized in the RGB color space, so 
and a code representing a centroid vector (hereinafter 
referred to as VQ code when appropriate), an error of 
the pixel value represented by the centroid vector cor- 
responding to the code relative to the original pixel value 
(hereinafter referred to as VQ residual when appropri- ss 
ate), and a codebook used for the vector quantization 
are output as coded data. 

[01 61] More specifically, image data to be encoded is 
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supplied to a frame menWy 101, and the frame memory 
101 sequentially stores image data supplied thereto, for 
example, on a frame basis. 

[01 62] A codebook creation unit 1 02 sequentially con- 
siders each frame of the image data stored in the frame 
memory 101, and creates a codebook to be used for 
vector quantization in the color space from the pixel val- 
ue of each of the pixels constituting the frame under con- 
sideration, for example, by what is called the LBG algo- 
rithm. The codebook is supplied to a vector quantization 
unit 103, and is output as (part of) coded data. 
[0163] The vector quantization unit 103 reads the 
frame under consideration from the frame memory 101, 
and sequentially considers each of the pixels constitut- 
ing the frame under consideration, for example, in order 
of raster scanning. The vector quantization unit 103 vec- 
tor-quantizes the pixel value of the pixel under consid- 
eration using the codebook supplied from the codebook 
creation unit 102, outputting the resulting VQ code and 
VQ residual as (part of) coded data. 
[0164] As shown in Fig. 22, in the encoding unit 91 
constructed as described above, the codebook creation 
un it 1 02 creates a codebook from the pixel value of each 
of the pixels constituting the frame under consideration, 
which is supplied to the vector quantization unit 1 03. The 
vector quantization unit 103 vector-quantizes the pixel 
value of a pixel under consideration using the codebook 
supplied from the codebook creation unit 102. 
[0165] That is, the vector quantization unit 103 de- 
tects, from the codebook, a centroid vector representing 
a point with the shortest distance to the point on the RGB 
space represented by the pixel value of the pixel under 
consideration, outputting a VQ code representing the 
centroid vector. Furthermore, the vector quantization 
unit 103 calculates the difference between the centroid 
vector represented by the VQ code and the vector cor- 
responding to the point on the RGB space represented 
by the pixel value of the pixel under consideration, out- 
putting the resulting difference vector as VQ residual. 
[0166] The encoding unit 91 then outputs the code- 
book, and the VQ code and VQ residual for each of the 
pixels in the frame under consideration, thus obtained, 
as coded data associated with the frame under consid- 
eration. 

[0167] Fig. 23 shows an example construction of the 
embedding unit 92 shown in Fig. 19, in a case where 
the encoding unit 91 is constructed as shown in Fig. 21 . 
[0168] The VQ code, VQ residual, and the codebook 
of the frame under consideration, output from the en- 
coding unit 91 , are supplied to and stored in a VQ code 
memory 111, a VQ residual memory 112, and a code- 
book memory 113, respectively. 
[01 69] The VQ code and the codebook stored respec- 
tively in the VQ code memory 111 and the codebook 
memory 113 are read therefrom, and supplied to a com- 
pression unit 115. 

[0170] A line rotation unit 114 sequentially considers 
each line in the frame under consideration, for example, 
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from top to bottom, and reads thWQ residual of the line 
under consideration stored in the VQ residual memory 
1 1 2. Furthermore, letting the number of pixels constitut- 
ing the line under consideration be denoted by x, the 
line rotation unit 114 receives additional information of 5 
the number of bits represented by int[log 2 x], and, for ex- 
ample, as shown in Fig. 24, rotates the VQ residual of 
the line under consideration to the right for the number 
of bits corresponding to the additional information, 
thereby embedding the additional information in the line 10 
under consideration. That is, with regard to a pixel on 
the line under consideration, the line rotation basically 
changes the VQ residual for the pixel from that obtained 
by the vector quantization, that is, the coding rule for the 
vector quantization is modified or destroyed. By destroy- '5 
ing the coding rule, the additional information is embed- 
ded. 

[0171] Then, the line rotation unit 114supplies the VQ 
residual of the line under consideration with the addi- 
tional information embedded therein to the compression 20 
unit 115. 

[01 72] The compression unit 1 1 5 compresses the VQ 
code, VQ residual, and codebook supplied thereto, for 
example, based on spatial correlation or bias of entropy, 
and outputs the results of the compression to a MUX 25 
116. The MUX 116 multiplexes the compressed VQ 
code, VQ residual, and codebook for output. 
[0173] Now, a process executed by the embedded 
compression encoder 11 shown in Fig. 19 (embedded 
coding process), in a case where the encoding unit 91 30 
is constructed as shown in Fig. 21 and the embedding 
unit 92 is constructed as shown in Fig. 23, will be de- 
scribed with reference to a flowchart shown in Fig. 25. 
[0174] In the encoding unit 91 (Fig. 21), a predeter- 
mined frame stored in the frame memory 1 01 is consid- 35 
ered, and in step S51 , each pixel in the frame under con- 
sideration is encoded by vector quantization, as de- 
scribed earlier. The VQ code, VQ residual, and code- 
book, constituting the coded data obtained by vector- 
quantizing the frame under consideration, are supplied *o 
to the embedding unit 92. 

[0175] In the embedding unit 92 (Fig. 23), the VQ 
code, VQ residual, and codebook associated with the 
frame under consideration, supplied from the encoding 
unit 91 , are supplied to and stored in the VQ code mem- *s 
ory 1 1 1 , the VQ residual memory 112, and the codebook 
memory 113, respectively. In step S52, the line rotation 
unit 114 determines whether all the lines of the frame 
under consideration for which a VQ residua! is stored in 
the VQ residual memory 112 have been processed. so 
[0176] If it is determined in step S52 that all the lines 
of the frame under consideration for which a VQ residual 
is stored in the VQ residual memory 112 have not been 
processed yet, the line rotation unit 114 selects the up- 
permost one of the lines yet to be processed in the frame 55 
under consideration as a new line to be considered. The 
process then proceeds to step S53. 
[0177] In step S53, the line rotation unit 114 rotates 
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each of the pixels in th^fffe under consideration, having 
VQ residuals as pixel values, to the right for the number 
of pixels corresponding to the additional information, 
thereby embedding the additional information in the line 
under consideration, and supplies the result to the com- 
pression unit 115. The process then returns to stepS52, 
repeating the same process. 

[0178] If it is determined in step S52 that ail the lines 
of the frame under consideration for which a residual is 
stored in the VQ residual memory 112 have been proc- 
essed, the process proceeds to step S54, in which the 
compression unit 115 reads the VQ code of the frame 
under consideration stored in the VQ code memory 111 , 
and also reads the codebook for the frame under con- 
sideration stored in the codebook memory 113. In step 
S54, the compression unit 1 1 5 compresses the VQ code 
and the codebook, and the VQ residual with the addi- 
tional information embedded therein, supplied from the 
line rotation unit 114, and supplies the results to the 
MUX 116. 

[0179] In step S55, the MUX 116 multiplexes the VQ 
code, VQ residual, and codebook supplied from the 
compression unit 115 for output. The process then pro- 
ceeds to step S56. 

[0180] In step S56, it is determined whether a next 
frame to be encoded in the encoding unit 91 exists. If it 
is determined that a next frame exists, with the next 
frame to be encoded as a new frame to be considered, 
the process returns to step S51 , repeating the same 
process. 

[0181] If it is determined in step S56 that no further 
frame exists to be encoded next, the embedded coding 
process is exited. 

[0182] Fig. 26 shows an example construction of the 
decoder 1 2 shown in Fig. 1 , in a case where the embed- 
ded compression encoder 11 is constructed as shown 
in Fig. 19. 

[0183] The embedded coded data output from the 
embedding unit 92 shown in Fig. 1 9 is supplied to a cod- 
ing rule restoring unit 121. 

[0184] The coding rule restoring unit 121 restores (de- 
codes) the embedded coded data into the coded data 
encoded according to the coding rule in the encoding 
unit 91 shown in Fig. 1 9, thereby decoding the additional 
information embedded in the embedded coded data. 
[0185] That is, the coding rule restoring unit 121 ma- 
nipulates the embedded coded data based on a param- 
eter supplied from a parameter controller 124, thereby 
obtaining a candidate of the coded data (hereinafter re- 
ferred to as tentative coded data when appropriate). 
Furthermore, the coding rule restoring unit 121 obtains 
a candidate of the additional information (hereinafter re- 
ferred to as tentative decoded additional information) 
embedded in the embedded coded data, based on an 
operation for restoring the embedded coded data into 
the tentative coded data. The tentative coded data is 
supplied to a decoding unit 122 and to a determination 
unit 1 23, while the tentative decoded additional informa- 
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tion is supplied to the determWf8n unit 123. 
[0186] The decoding unit 122 performs a decoding 
operation upon the tentative coded data supplied from 
the coding rule restoring unit 121 , based on the coding 
rule in the encoding unit 91 shown in Fig. 19, thereby 
obtaining a candidate of the original pixel value (herein- 
after referred to as tentative decoded pixel value when 
appropriate). The tentative decoded pixel value is sup- 
plied to the determination unit 123. 
[01 87] The determination unit 1 23 controls the param- 
eter controller 1 24 to supply one or more parameter val- 
ues to the coding rule restoring unit 121, and determines 
a correct tentative decoded pixel value (coinciding with 
the original pixel value) from one or more the tentative 
decoded pixel values obtained respectively in accord- 
ance with the one or more parameter values. Further- 
more, the determination unit 123 selects, from one or 
more tentative decoded additional information supplied 
from the coding rule restoring unit 121 respectively in 
association with the one or more tentative decoded pixel 
values, the tentative decoded additional information as- 
sociated with the correct decoded pixel value as correct 
decoded additional information, outputting the correct 
decoded pixel value and the tentative decoded addition- 
al information as the final results of decoding of pixel 
value and additional information (decoded pixel value 
and decoded additional information), respectively. 
[0188] The parameter controller 124, under the con- 
trol of the determination unit 123, supplies a parameter 
for manipulating the embedded coded data to th e coding 
rule restoring unit 121. 

[0189] Now, a process executed by the decoder 12 
shown in Fig. 26 (decoding process) will be described 
with reference to a flowchart shown in Fig. 27. 
[01 90] Initially, in step S61 , the determination unit 1 23 
controls the parameter controller 124 to set a parameter 
to be supplied to the coding rule restoring unit 121 . Ac- 
cordingly, the parameter controller 126 supplies a pa- 
rameter, in accordance with the control by the determi- 
nation unit 123, to the coding rule restoring unit 121. 
[0191] In step S62, the coding rule restoring unit 121 
manipulates the embedded coded data based on the 
parameter supplied from the parameter controller 124, 
translating it into the tentative coded data, and supplies 
the tentative coded data to the decoding unit 1 22 and to 
the determination unit 1 23. Furthermore, the coding rule 
restoring unit 121 , based on the operation for restoring 
the embedded coded data into tentative coded data, that 
is, the parameter supplied from the parameter controller 
124, decodes the additional information embedded in 
the embedded coded, and supplies the result to the de- 
termination unit 123 as tentative decoded additional in- 
formation. 

[01 92] In step S63, the decoding unit 1 22 decodes the 
tentative coded data supplied from the coding rule re- 
storing unit 1 21 based on the coding rule in the encoding 
unit 91 shown in Fig. 1 9, and supplies the resulting pixel 
value to the determination unit 123 as tentative decoded 
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[0193] In step 564, the determination unit 123 deter- 
mines whether the tentative decoded pixel value sup- 
plied from the decoding unit 122 is a correct decoding 

5 result (coinciding with the original pixel value). If it is de- 
termined as not coinciding, the process returns to step 
S61 . In this case, in step S61 . the determination unit 1 23 
sets a new value as a parameter to be output from the 
parameter controller 124, and the same process is re- 

10 peated. 

[0194] If it is determined in step S64 that the tentative 
decoded pixel value is a correct decoding result, the 
process proceeds to step 565, in which the determina- 
tion unit 123 outputs the tentative decoded pixel value 

15 as decoded pixel value constituting the result of decod- 
ing of the original pixel value. Furthermore, the determi- 
nation unit 123 outputs the tentative decoded additional 
information associated with the decoded pixel value, as 
decoded additional information constituting the result of 

20 decoding of the additional information embedded. The 
process then proceeds to step S66. 
[0195] In step S66, it is determined whether further 
embedded coded data to be decoded is remaining. If it 
is determined as remaining, the process returns to step 

25 S61 , repeating the same process for the embedded cod- 
ed data to be decoded. 

[0196] If it is determined in step S66 that further em- 
bedded coded data to be decoded does not exist, the 
decoding process is exited. 
30 [0197] Fig. 28 shows an example construction of the 
coding rule restoring unit 121 shown in Fig. 26, in a case 
where the embedding unit 92 shown in Fig. 19 is con- 
structed as shown in Fig. 23. 

[0198] Data output from the MUX 116 shown in Fig. 

35 23 is supplied to a DEMUX 131. The DEMUX 131 de- 
multiplexes the data supplied thereto into compressed 
VQ code, VQ residual, and codebook, and supplies 
them to an expansion unit 132. The expansion unit 132 
expands the compressed VQ code, VQ residual, and 

^o codebook supplied from the DEMUX 131 , and supplies 
the expanded VQ code, VQ residual, and codebook to 
a VQ code memory 133, a VQ residual memory 134, 
and to a codebook memory 135, respectively. 
[0199] The VQ code memory 133, the VQ residual 

^5 memory 134, and the codebook memory 135 store, on 
a frame basis, the VQ code, VQ residual, and codebook 
supplied from the expansion unit 132, respectively. 
[0200] A line rotation unit 136 sequentially considers 
each line of the frame stored in the VQ residual memory 

50 1 34, for example, from top to bottom, and reads the VQ 
residual of the line under consideration stored in the VQ 
residual memory 134. Furthermore, letting the number 
of pixels constituting the line under consideration be de- 
noted by x, the line rotation unit 136 receives an integer 

55 in a range of zero to x as a parameter from the parameter 
controller 124 (Fig. 26), and rotates the VQ residual of 
the line under consideration to the left for the number of 
pixels corresponding to the parameter. The line rotation 
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unit 136 outputs the VQ resid^pCf each line after the 
rotation as tentative coded data, together with the VQ 
code stored in the VQ code memory 133 and the code- 
book stored in the codebook memory 135. 
[0201] Furthermore, the line rotation unit 136 outputs 
the parameter value supplied from the parameter con- 
troller 124 as tentative decoded additional information. 
[0202] Fig. 29 shows an example construction of the 
decoding unit 122 shown in Fig. 26, in a case where the 
encoding unit 91 shown in Fig. 19 is constructed as 
shown in Fig. 21. 

[0203] The decoding unit 1 22 decodes pixel values by 
inverse vector quantization based on the VQ code, VQ 
residual, and codebook of a frame, constituting the cod- 
ed data supplied from the coding rule restoring unit 121. 
[0204] That is, to an inverse vector quantization unit 

141, the VQ code and codebook are supplied. The in- 
verse vector quantization unit 141 detects a centroid 
vector corresponding to the VQ code from the code- 
book, and supplies the centroid vectorto an addition unit 

142. To the addition unit 142, as well as the centroid 
vector supplied from the inverse vector quantization unit 
141, a difference vector constituting the VQ residual is 
also supplied. The addition unit 142 adds the centroid 
vector and the difference vector. The addition unit 142 
outputs pixel values having the components of the vec- 
tor obtained by the addition as the R, G, and B values 
as tentative decoded pixel value. 

[0205] Fig. 30 shows an example construction of the 
determination unit 1 23 shown in Fig. 26, in a case where 
the coding rule restoring unit 1 21 and the decoding unit 
122 are constructed as shown in Figs. 28 and 29, re- 
spectively. 

[0206] To a memory 1 51 , the tentative decoded addi- 
tional information from the coding rule restoring unit 1 21 
and the tentative decoded pixel value from the decoding 
unit 122 are supplied. The memory 151 temporarily 
stores the tentative decoded additional information and 
the tentative decoded pixel value, and reads the tenta- 
tive decoded additional information and the tentative de- 
coded pixel value under the control of a true/false de- 
termination unit 154, outputting them as decoded addi- 
tional information and decoded pixel value, respectively. 
[0207] To an encoding unit 1 52, the codebook includ- 
ed in the tentative coded data output from the coding 
rule restoring unit 121 and the tentative decoded pixel 
value output from the decoding unit 122 are supplied. 
The encoding unit 152 encodes the tentative decoded 
pixel value similarly to the encoding unit 91 shown in 
Fig. 19. That is, the encoding unit 152 vector-quantizes 
the tentative decoded pixel value using the codebook 
supplied from the coding rule restoring unit 1 21 , supply- 
ing the resulting VQ code and VQ residual to a compar- 
ison unit 153. The VQ code and VQ residual obtained 
by vector-quantizing the tentative decoded pixel value 
will hereinafter be referred to, when appropriate, as ten- 
tative VQ code and tentative VQ residual, respectively. 
[0208] To the comparison unit 1 53, as well as the ten- 



30 



tative VQ code and IlBRive VQ residual output from 
the encoding unit 1 52, the VQ code and VQ residual in- 
cluded in the tentative coded data output from the cod- 
ing rule restoring u nit 1 21 are supplied. The comparison 

5 unit 153 compares the tentative VQ code with the VQ 
code in the tentative coded data, and also compares the 
tentative VQ residual with the VQ residual in the tenta- 
tive coded data, supplying the results of the compari- 
sons to the true/false determination unit 154. 

w [0209] The true/false determination unit 1 54 controls 
the parameter controller 124 to supply the number of 
bits for line rotation to the coding rule restoring unit 121 
as a parameter. Furthermore, the true/false determina- 
tion unit 1 54 determines whether the tentative decoded 

15 pixel value is a correct decoding result based on the re- 
sult of the comparison of the tentative VQ code and the 
VQ code in the tentative coded data, and the result of 
the comparison of the tentative VQ residual and the VQ 
residual in the tentative coded data, supplied from the 

20 comparison unit 153, and controls reading of the tenta- 
tive decoded pixel value and the tentative decoded ad- 
ditional information from the memory 151 based on the 
result of the determination. 

[0210] Now, the principle of determination by the true/ 
25 false determination unit 154 shown in Fig. 30 as to 
whether the tentative decoded pixel value is a correct 
decoding result will be described with reference to Fig. 
31. 

[0211] Since the encoding unit 91 (Fig. 19) performs 

30 vector quantization in the RGB space, a centroid vector 
used in the vector quantization is composed of three 
components, namely, R, B, and G components. Letting 
the centroid vector composed of the R, B, and G com- 
ponents be denoted as (R, G, B), and assuming that 

35 each of the R, B, and G components of the centroid vec- 
tor in the codebook is represented by a multiple of ten 
for simplicity of description, for example, a pixel value 
with R, B, and G components of 102, 103, and 99, re- 
spectively, (hereinafter denoted as pixel value (102, 

40 103, 99) when appropriate) has the shortest distance 
relative to a centroid vector (100, 100, 100), and thus is 
vector-quantized into a VQ code associated with the 
centroid vector (1 00, 1 00, 1 00). For example, let the VQ 
code associated with the centroid vector (1 00, 1 00, 1 00) 

45 beO. 

[0212] In this case, subtracting the centroid vector 
(1 00, 1 00, 1 00) from the pixel value (1 02, 1 03, 99) yields 
the VQ residual (2, 3, -1). Thus, the pixel value (102, 
103, 99) is encoded into the VQ code 0 and the VQ re- 

so sidual (2, 3, -1). 

[0213] By inversely vector-quantizing the VQ code 0 
and the VQ residual (2, 3, -1), constituting the coded 
data obtained in the encoding unit 91 , as shown in Fig. 
31(A), the centroid vector (100, 100, 100) associated 

55 with the VQ code 0 and the VQ residual (2, 3, -1) are 
added to yield the pixel value (102, 103, 99), and thus 
correctly decoded to obtain the original pixel value. 
[0214] Furthermore, by vector-quantizing the decod- 
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ed pixel value (1 02, 1 03, 99) afPr!, as shown in Fig. 31 
(A), the VQ code 0 and the VQ residual (2, 3, -1) are 
obtained again. 

[0215] From the above, by decoding the VQ code and 
the VQ residual constituting the coded data to obtain 5 
correct decoding result, and re-encoding (vector quan- 
tization herein) the decoding result, the VQ code and 
the VQ residual obtained by the encoding coincide with 
the VQ code and the VQ residual constituting the coded 
data, respectively. 10 
[0216] Of the VQ code 0 and VQ residual (2, 3, -1) 
constituting the coded data obtained in the encoding unit 
91 , when additional information is embedded by apply- 
ing line rotation on the VQ residual (2, 3, -1 ) in the man- 
ner described earlier, VQ residual obtained for a differ- 15 
ent pixel is assigned as VQ residual between the VQ 
code 0 and the pixel associated with the VQ residual (2, 
3, -1). If, for example, the VQ residual of the different 
pixel is (10, 11, 12) as shown in Fig. 31(B), by adding 
the centroid vector (1 00. 1 00, 1 00) and the VQ residual 20 
(1 0, 1 1 , 1 2), the VQ code 0 and the VQ residual (1 0, 1 1 , 
1 2) are decoded as the pixel value (1 1 0, 1 1 1 . 1 1 2), and 
are not decoded correctly into the original pixel value 
(102, 103, 99). 

[0217] Thus, even if the incorrect decoded pixel value 25 
(110, 111, 112) is vector-quantized again, the resulting 
VQ code and VQ residual do not coincide with the VQ 
code 0 and the VQ residual (2, 3, -1) constituting the 
coded data, respectively, as shown in Fig. 31(B). 
[0218] More specifically, in this case, since it is as- 30 
sumed that each of the R, B, and G components of a 
centroid vector in the codebook is represented by a mul- 
tiple of ten , the centroid vector with the shortest distance 
to the pixel value (1 1 0, 1 1 1 , 1 1 2) is (1 1 0, 1 1 0, 1 1 0). Thus, 
for example, assuming that the VQ code representing 35 
the centroid vector (1 1 0, 1 1 0, 1 1 0) is 1 , the pixel value 
(110, 111, 112) is vector-quantized into the VQ code 1 
and the VQ residual (0, 1, 2) (= (110, 111, 112) -(110, 
110, 110)). In this case, neither the VQ code northe VQ 
residual coincides with the VQ code or the VQ residual *o 
(10. 11 , 12) constituting the original coded data. 
[0219] From the above, if the number of pixels for the 
rotation according to the parameter in the line rotation 
unit 1 36 shown in Fig. 28 does not coincide with the ad- 
ditional information, that is, if the tentative coded data ^ 
does not coincide with the coded data before the addi- 
tional information is embedded therein, VQ code and 
VQ residual obtained by re-encoding the tentative de- 
coded pixel value obtained from the tentative coded da- 
ta do not coincide respectively with the VQ code and the so 
VQ residual constituting the tentative coded data, and 
thus the tentative decoded pixel value obtained by de- 
coding the tentative coded data can be determined as 
incorrect decoding result. 

[0220] On the other hand, if the number of pixels for 55 
the rotation according to the parameter in the line rota- 
tion unit 136 shown in Fig. 28 coincides with the addi- 
tional information, that is, if the tentative coded data co- 
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incides with the cod^lWta before the additional infor- 
mation is embedded therein, VQ code and VQ residual 
obtained by re-encoding the tentative decoded pixel val- 
ue obtained from the tentative coded data coincide re- 
spectively with the VQ code and the VQ residual consti- 
tuting the tentative coded data, and thus the tentative 
decoded pixel value obtained by decoding the tentative 
coded data can be determined as correct decoding re- 
sult. 

[0221] Now, a decoding process executed by the de- 
coder 12 shown in Fig. 26, in a case where the coding 
rule restoring unit 121, the decoding unit 122, and the 
determination unit 123 are constructed as shown in 
Figs. 28 to 30, respectively, will be described with refer- 
ence to a flowchart shown in Fig. 32. 
[0222] In the decoding process, the DEMUX 131 of 
the coding rule restoring unit 121 demultiplexes data 
supplied thereto into compressed VQ code, VQ residu- 
al, and codebook, supplying them to the expansion unit 
132. In step S71. the expansion unit 132 expands the 
compressed VQ code, VQ residual, and codebook sup- 
plied from the DEMUX 1 31 , supplying the expanded VQ 
code, VQ residual, and codebook to and stores them in 
the VQ code memory 1 33, the VQ residual memory 1 34, 
and the codebook memory 135, respectively. 
[0223] Then, in step S72, the true/false determination 
unit 154 of the determination unit 123 (Fig. 30) controls 
the parameter controller 1 24 (Fig. 26) to set a particular 
parameter value, and supplies the parameter to the cod- 
ing rule restoring unit 121 . 

[0224] The true/false determination unit 154, for ex- 
ample, sequentially sets integers in a range of zero to 
the number of pixels on one line as the parameter value, 
each time the process in step S72 is executed for each 
line of each frame. 

[0225] When the coding rule restoring unit 121 re- 
ceives the parameter from the parameter controller 124, 
in step S73, the line rotation unit 136 rotates the VQ re- 
sidual of the line under consideration to the left for the 
number of pixels corresponding to the parameter, sup- 
plying the VQ residual having been left-rotated, together 
with the VQ code of the line under consideration stored 
in the VQ code memory 133 and the codebook stored 
in the codebook memory 1 35, to the decoding unit 1 22 
as tentative coded data. Furthermore, the line rotation 
unit 1 36 supplies the number of pixels for which the line 
under consideration is rotated to the determination unit 
123 as tentative decoded additional information. 
[0226] In step S74, the decoding unit 122 performs 
inverse vector quantization based on the VQ code, VQ 
residual, and codebook constituting the tentative coded 
data supplied from the coding rule restoring unit 121, 
thereby decoding the pixel values of the line under con- 
sideration, and supplies the resulting tentative decoded 
pixel values to the determination unit 123. 
[0227] In the determination unit 1 23 (Fig. 30), the ten- 
tative decoded additional information output from the 
coding rule restoring unit 1 21 and the tentative decoded 
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pixel values output from the bSSrig rule restoring unit 
121 are stored in the memory 151 . Furthermore, in step 

575, in the determination unit 1 23, the encoding unit 1 52 
vector-quantizes the tentative decoded pixel values 
supplied from the decoding unit 122, and supplies the 5 
resulting tentative VQ code and tentative VQ residual to 
the comparison unit 153. 

[0228] The comparison unit 153 compares, for each 
pixel of the line under consideration, the tentative VQ 
code supplied from the encoding unit 152 with the VQ 
code constituting the tentative coded data, and also 
compares the tentative VQ residual supplied from the 
encoding unit 1 52 with the VQ residual constituting the 
tentative coded data, supplying the results of the com- 
parisons to the true/false determination unit 154. In step 

576, the true/false determination unit 154 determines, 
for each pixel of the line under consideration, whether 
the tentative VQ code coincides with the VQ code con- 
stituting the tentative coded data, and whether the ten- 
tative VQ residual coincides with the VQ residual con- 
stituting the tentative coded data. If it is determined that 
one or neither do not coincide for one or more pixels, 
that is, if the value of the parameter set in the immedi- 
ately preceding step S72 does not coincide with the ad- 
ditional information, the process returns to step S72, in 
which a new parameter value is set, and the same proc- 
ess is repeated. 

[0229] If it is determined in step S76 that the tentative 
VQ code coincides with the VQ code constituting the 
tentative coded data and the tentative VQ residual co- 
incides with the VQ residual constituting the tentative 
coded data for every pixel of the line under considera- 
tion, that is, if the parameter value set in the immediately 
preceding step S72 coincides with the additional infor- 
mation and if the coded data has been restored and the 
original additional information has been correctly decod- 
ed as the tentative decoded additional information, the 
process proceeds to step S77, in which the true/false 
determination unit 154 controls the memory 151 to out- 
put the tentative decoded pixel value and the tentative 
decoded additional information of each pixel of the line 
under consideration, stored therein, as correct decoded 
pixel value and decoded additional information, respec- 
tively. The process then proceeds to step S78. 
[0230] In step S78, it is determined whether a line to 
be decoded next exists. If it is determined that a next 
line exists, the process returns to step S72, in which the 
same process is repeated with the next line to be de- 
coded as a new line to be considered. 
[0231] If it is determined in step S78that no line exists 
to be decoded next, the decoding process is exited. 
[0232] The decoding process shown in Fig. 32 is ex- 
ecuted for each frame. 

[0233] As described above, a coding rule for obtaining 
coded data is destroyed (modified) based on additional 
information, and embedded coded data encoded ac- 
cording to the destroyed coding rule is manipulated in a 
predetermined manner, whereby tentative coded data 



34 

is obtained. The tentlt^Sxoded data is decoded, and it 
is determined whether encoding the result of decoding 
yields the same data as the tentative coded data, so that 
coded data prior to destruction of the coding rule is re- 
stored (decoded). Accordingly, by utilizing the destruc- 
tion and restoration of the coding rule, additional infor- 
mation can be embedded and decoded without increas- 
ing the amount of coded data. 

[0234] The applicant has already proposed, in United 
States Patent Application No. 09/636138, a method for 
embedding additional information in an image based on, 
for example, correlation of images. According to the pro- 
posed method, for example, lines constituting a frame 
are exchanged based on additional information, and the 
lines of the frame that have been exchanged are re- 
stored to the original positions based on the fact that 
adjacent lines in the original frame are highly correlated 
with each other. Depending on cases, the method based 
on correlation has not always been successful in em- 
bedding additional information. That is, simply put, the 
method based on correlation, with regard to a line of a 
frame in which additional information is embedded, ex- 
changes a line adjacent to the line with a line that is most 
highly correlated with the line under consideration, until 
all the lines are restored to original positions. However, 
in some images, a line that is most highly correlated with 
a line under consideration is not necessarily a line to be 
positioned adjacent to the line under consideration. 
When such lines are exchanged by embedding addition- 
al information, the lines cannot be restored to original 
positions based on correlation, inhibiting additional in- 
formation from being embedded. 
[0235] As opposed thereto, the method described 
earlier is not subject to such a failure of decoding. 
[0236] In the embodiment shown in Fig. 26, the de- 
coder 12 includes only a single line of the pair of the 
coding rule restoring unit 121 and the decoding unit 1 22, 
sequentially obtaining tentative coded data associated 
with various parameter values by sequentially changing 
parameters output from the parameter controller 124; 
however, for example, as shown in Fig. 33, the decoder 
12 may include M lines of coding rule restoring units 
1 21 1 to 1 21 M and decoding units 1 22 1 to 1 22 M , supply- 
ing different parameter values respectively to the each 
coding rule restoring units 121 m (m = 1, 2, *••, M) so that 
tentative coded data respectively associated with the 
parameter values can be simultaneously obtained. 
[0237] Furthermore, although the encoding unit 91 
shown in Fig. 1 9 performs vector quantization in the ex- 
ample described above, the encoding unit 91 may per- 
form, for example, predictive coding instead of vector 
quantization. In that case, the embedding unit 92 is al- 
lowed to embed additional information by manipulating 
prediction residual associated with predictive coding 
based on the additional information. 
[0238] Furthermore, although additional information 
is embedded by rotating VQ residual of each line based 
on the additional information in the example described 
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above, additional information mty also be embedded, 
for example, by rotating a bit sequence of a value rep- 
resenting the VQ residual. 

[0239] Furthermore, although codebooks are created 
and included in coded data on a frame basis in the ex- 5 
ample described above, codebooks may be created in 
advance and stored in each of the embedded compres- 
sion encoder 11 and the decoder 12. Furthermore, 
codebooks may be created on a basis of a plurality of 
frames, or on a basis of a predetermined area of a frame. 10 
[0240] The series of processes described above may 
be implemented in hardware or in software. When the 
series of processes is implemented in software, a pro- 
gram constituting the software is installed, for example, 
on a general-purpose computer. 15 
[0241] Fig, 34 shows an example construction of a 
computer on which a program for executing the series 
of processes described above is installed. 
[0242] The program may be stored in advance in a 
recording medium incorporated in the computer, such 20 
as a hard disc 205 or a ROM 203. 
[0243] Alternatively, the program may be temporarily 
or permanently stored (recorded) on a removable stor- 
age medium 211 such as a floppy disk, a CD-ROM 
(Compact Disc Read Only Memory), an MO (Magneto 25 
optical) disc, a DVD (Digital Versatile Disc), a magnetic 
disc, or a semiconductor memory. The removable re- 
cording medium 21 1 can be provided in the form of what 
is called package software. 

[0244] Instead of installing the program on the com- 30 
puter from the removable recording medium 211 as de- 
scribed above, the program may be transferred by wire- 
less to the computer from a downloading site via an ar- 
tificial satellite for digital satellite broadcasting, or may 
be transferred to the computer by wired link via a net- 35 
work such as a LAN (Local Area Network) or the Inter- 
net, so that the computer receives the program thus 
transferred via a communication unit 208 and then in- 
stalls the program in an internal hard disc 205. 
[0245] The computer includes a CPU (Central 40 
Processing Unit) 202. To the CPU 202, an input/output 
interface 210 is connected via a bus 201. In response 
to a command input via the input/output interface 210 
by a user's operation on an input unit 207 including a 
keyboard, a mouse, a microphone, etc., the CPU 202 45 
executes a program stored in a ROM (Read Only Mem- 
ory) 203 according to the command. Alternatively, the 
CPU 202 loads a program stored in the hard disc 205, 
a program transferred via a satellite or a network, re- 
ceived via the communication unit 208, and installed on so 
the hard disc 205, or a program read from the removable 
recording medium 211 mounted on a drive 209 and in- 
stalled on the hard disc 205, into a RAM (Random Ac- 
cess Memory) 204 for execution. The CPU 202 thus ex- 
ecutes processes according to the flowcharts described 55 
above, or processes executed by the constructions 
shown in the block diagrams described above. The CPU 
202, as required, outputs the results of the processes to 
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an output unit 206 inclWPg an LCD (Liquid Crystal Dis- 
play), a speaker, etc., transmits them via the communi- 
cation unit 208, records them on the hard disc 205, etc., 
for example, via the input/output interface 210. 
[0246] In this specification, the processing steps con- 
stituting programs for executing various processes on a 
computer need not necessarily be executed sequential- 
ly in the order shown in the flowcharts, and may be ex- 
ecuted in parallel or individually (e.g., parallel process- 
ing or processes based on objects). 
[0247] The program may be executed by a single 
computer or by distributed processing using a plurality 
of computers. Furthermore, the program may be trans- 
ferred to and executed on a remote computer. 
[0248] Although image data is encoded in the embod- 
iment, data to be encoded is not limited to image data, 
and various data such as audio data and computer pro- 
grams may be used. 

Industrial Applicability 

[0249] According to the first data processing appara- 
tus, data processing method, and storage medium of the 
present invention, first data is encoded to output coded 
data. Then, a portion of the coded data is modified 
based on the second data so that the second data is 
embedded in the coded data. 

[0250] According to the second data processing ap- 
paratus, data processing method, and storage medium 
of the present invention, a coding table for encoding first 
data is created, and the coding table created is modified 
based on second data to create a modified coding table. 
Then, the first data'is encoded based on the modified 
coding table to generate embedded coded data in which 
the second data is embedded. 
[0251 ] According to the fifth data processing appara- 
tus, data processing method, and storage medium of the 
present invention, first data is encoded according to a 
coding rule to output coded data. Then, the coding rule 
is modified based on second data, and the first data is 
encoded according to the modified coding rule to gen- 
erate embedded coded data in which the second data 
is embedded. 

[0252] Thus, the second data can be embedded with- 
out increasing the amount of data. 
[0253] According to the third data processing appara- 
tus, data processing method, and storage medium of the 
present invention, embedded coded data encoded by 
embedding second data in first data is tentatively de- 
coded based on a coding table to output tentative de- 
coded data. Furthermore, a tentative coding table is cre- 
ated based on the tentative decoded data, and the em- 
bedded coded data is decoded based on the coding ta- 
ble and the tentative coding table to obtain first decoded 
data. The coding table is compared with the tentative 
coding table to obtain second decoded data. 
[0254] According to the fourth data processing appa- 
ratus, data processing method, and storage medium of 
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the present invention, a portiorWKmbedded coded da- 
ta encoded by embedding second data in first data is 
modified according to an input parameter, and tentative- 
ly decoded based on a coding table to output tentative 
decoded data. Furthermore, the tentative decoded data 
is re-encoded to output re-encoded data. The parameter 
is determined by comparing the embedded coded data 
and the re-encoded data, and tentative decoded data 
obtained by tentatively decoding, based on the coding 
table, the embedded coded data having been partially 
modified based on the parameter, is output as first de- 
coded data, and for second decoded data correspond- 
ing to the parameter is also obtained. 
[0255] According to the sixth data processing appa- 
ratus, data processing method, and storage medium of 
the present invention, embedded coded data obtained 
by embedding second data in first data is decoded to 
obtain coded data encoded according to an entropy cod- 
ing rule and to obtain the second data. Furthermore, the 
coded data is decoded to obtain the first data, 
[0256] Thus, the first and the second data can be cor- 
rectly decoded. 

[0257] According to the data processing system of the 
present invention, in the encoding apparatus, first data 
is encoded according to a coding rule to output coded 
data. Then , the coding rule is modified based on second 
data, and the first data is encoded according to the mod- 
ified coding rule to generate embedded data in which 
the second data is embedded. In the decoding appara- 
tus, embedded coded data is decoded to obtain the cod- 
ed data encoded according to the coding rule and to ob- 
tain the second data. Then, the coded data is decoded 
to obtain the first data. Thus, the second data can be 
embedded without increasing the amount of coded data 
obtained by encoding the first data. Furthermore, the da- 
ta in which the second data is embedded can be cor- 
rectly decoded to obtain the first and the second data. 



Claims 

1 . A data processing apparatus comprising: 

encoding means for encoding first data to out- 
put coded data; and 

embedding means for embedding second data 
in the coded data output from said encoding 
means by modifying a portion of the coded data 
based on the second data. 

2. A data processing apparatus according to Claim 1 , 

wherein said encoding means comprises: 

coding table creation means for statistically an- 
alyzing data values included in the first data to 
create a coding table for encoding the data val- 
ues into coded data; and 
coded data output means for generating and 
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o utputting coSiWata by encoding the first data 
based on the coding table created by said cod- 
ing table creation means. 

5 3. A data processing apparatus according to Claim 2, 
wherein said embedding means comprises: 

translation table creation means for creating a 
translation table for translating coded data in 

10 the coding table created by said coding table 

creation means, based on the second data; and 
embedded coded data generation means for 
translating the coded data output from said cod- 
ed data output means based on the translation 

15 table created by said translation table creation 

means to generate embedded coded data in 
which the second data is embedded. 

4. A data processing apparatus according to Claim 3, 
20 wherein said coding table creation means 

comprises: 

frequency table creation means for creating 
and outputting a frequency table representing 
25 the frequency of each data value included in the 

first data; 

Huffman tree creation means for creating 
and outputting a Huffman tree based on the 

30 frequency table output from said the fre- 

quency table creation means; and 
Huffman table creation means for creating 
a Huffman table associating each of the da- 
ta values with the coded data, based on the 

35 Huffman tree created by said Huffman tree 

creation means; 

and said coded data output means gen- 
erates and outputs coded data by encoding the 
to first data based on the Huffman table created 

by said Huffman table creation means. 



A data processing apparatus according to Claim 4, 
wherein said translation table creation means 
comprises: 



45 



pattern number detection means for detecting 
the number of patterns of coded data of each 
code length in the Huffman table created by 

50 said Huffman table creation means; and 

information amount detection means for detect- 
ing the amount of information that can be em- 
bedded in coded data of each code length in 
the Huffman table, based on the number of pat- 

55 terns detected by said pattern number detec- 

tion means; 

and said translation table creation means cre- 
ates and outputs a translation table in which 
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coded data of each co^P&ngth in the Huffman 
table having been modified based on a portion 
of the second data in accordance with the 
amount of information detected by said infor- 
mation amount detection means is associated 5 
with coded data of each code length priorto the 
modification. 

6. A data processing apparatus according to Claim 5, 

wherein said embedded coded data genera- 10 
tion means comprises code length detection means 
for detecting the code length of coded data under 
consideration in the coded data output from said 
coded data output means; 

and said embedded coded data generation *5 
means translates the coded data under considera- 
tion into embedded coded data for output based on 
the translation table associated with the code length 
detected by said code length detection means. 

20 

7. A data processing apparatus according to Claim 2, 

wherein said coded data output means com- 
prises quantization means for outputting a quanti- 
zation code generated by quantizing a data value 
included in the first data based on the coding table, 25 
and a quantization error representing an error be- 
tween the quantization code and the data value. 

8. A data processing apparatus according to Claim 7, 

wherein said embedding means embeds the 30 
second data in the coded data output from said 
quantization means by modifying a portion of the 
quantization error based on the second data. 

9. A data processing apparatus according to Claim 8, 35 

wherein said coding table creation means 
comprises codebook creation means for creating 
and outputting a codebook including a vector quan- 
tization code representing a representative vector 
of a data value included in the first data; 40 

said quantization means comprises vector 
quantization means for vector-quantizing each data 
value included in the first data based on the code- 
book created by said codebook creation means to 
output a vector quantization code, and for detecting ^ 
and outputting a vector quantization error repre- 
senting the difference between the data value and 
the representative vector represented by the vector 
quantization code; 

and said embedding means embeds the sec- so 
ond data in the coded data output from said vector 
quantization means by modifying a portion of the 
vector quantization error based on the second data. 

10. A data processing apparatus according to Claim 8, 55 

wherein said embedding means embeds the 
second data in the coded data output from said 
quantization means by rotating a portion of the 
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quantization erro^Ked on the second data. 

11. A data processing apparatus comprising: 

coding table creation means for creating a cod- 
ing table for encoding first data; 
modified coding table creation means for mod- 
ifying the coding table created by said coding 
table creation means based on second data to 
create a modified coding table; and 
embedded coded data generation means for 
encoding the first data based on the modified 
coding table to generate embedded coded data 
in which the second data is embedded. 

12. A data processing apparatus according to Claim 11, 

wherein said coding table creation means sta- 
tistically analyzes data values included in the first 
data to create the coding table for encoding the data 
values into coded data. 

13. A data processing apparatus according to Claim 12, 

wherein said modified coding table creation 
means comprises translation table creation means 
for creating a translation table for translating coded 
data in the coding table created by said coding table 
creation means based on the second data, 

and said modified coding table creation 
means creates a modified coding table by translat- 
ing a portion of the coding table based on the trans- 
lation table created by said translation table crea- 
tion means. 

14. A data processing apparatus according to Claim 13, 

wherein said coding table creation means 
comprises: 

frequency table creation means for creating 
and outputting a frequency table representing 
a frequency of each data value included in the 
first data; 

Huffman tree creation means for creating 
and outputting a Huffman tree based on the 
frequency table output from said frequency 
table creation means; and 
Huffman table creation means for creating 
a Huffman table associating each data val- 
ue with the coded data, based on the Huff- 
man tree created by said Huffman tree cre- 
ation means; 

and said modified coding table creation 
means creates a modified Huffman table by 
translating a portion of the Huffman table cre- 
ated by said Huffman table creation means. 

1 5. A data processing apparatus according to Claim 1 4, 
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wherein said translal 
comprises: 
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ble creation means 



pattern number detection means for detecting 
the number of patterns of coded data of each 5 
code length in the Huffman table created by 
said Huffman table creation means; and 
information amount detection means for detect- 19. 
ing the amount of information that can be em- 
bedded in coded data of the code length in the 10 
Huffman table, based on the number of pat- 
terns detected by said pattern number detec- 
tion means; 

and said translation table creation means cre- 
ates and outputs a translation table in which ^ 
coded data of each code length in the Huffman 
table having been modified based on a portion 
of the second data in accordance with the 
amount of information detected by said infor- 
mation amount detection means is associated 20 
with coded data of each code length prior to the 
modification. 

1 6. A data processing apparatus according to Claim 1 5, 

wherein said embedded coded data genera- 25 
tion means translates the first data into embedded 
coded data for output based on the modified Huff- 20. 
man table created by said modified coding table 
creation means. 

30 

17. A data processing apparatus comprising: 

tentative decoding means for tentatively decod- 
ing embedded coded data encoded by embed- 
ding second data in first data, based on a cod- 35 
ing table, to output tentative decoded data; 
tentative coding table creation means for cre- 
ating a tentative coding table based on the ten- 
tative decoded data; 

first decoded data obtaining means for decod- 40 
ing the embedded coded data based on the 
coding table and the tentative coding table to 
obtain first decoded data; and 
second decoded data obtaining means for ob- 
taining second decoded data by comparing the 45 
coding table with the tentative coding table. 



A1 ^^42 

man tree creSIRy said Huffman tree creation 
means; 

and said tentative decoding means de- 
codes the embedded coded data based on the 
Huffman table to output embedded decoded 
data. 

A data processing apparatus according to Claim 18, 
wherein said tentative coding table creation 
means comprises: 

further frequency table creation means for cre- 
ating a further frequency table representing the 
frequency of occurrence of each data value in 
the embedded decoded data based on the em- 
bedded decoded data; 

further Huffman tree creation means for creat- 
ing a further Huffman tree based on the further 
frequency table created by said further frequen- 
cy table creation means; and 
further Huffman table creation means forcreat- 
ing a further Huffman table associating a data 
value in the first data with embedded coded da- 
ta, based on the further Huffman tree created 
by said further Huffman tree creation means. 

A data processing apparatus according to Claim 19, 
further comprising: 

comparison means for comparing the frequen- 
cies in the frequency table and the further fre- 
quency table to create a data value association 
table associating the same data values with 
each other; and 

translation table creation means for creating a 
modified table associating coded data and em- 
bedded coded data respectively associated 
with the data values associated with each other 
by the data value association table created by 
said comparison means, based on the Huffman 
table and the further Huffman table; 

wherein said second decoded data obtaining 
means obtains the second decoded data based on 
the association between the coded data and the 
embedded coded data in the translation table. 



1 8. A data processing apparatus according to Claim 1 7, 
wherein said tentative decoding means com- 
prises: 

Huffman tree creation means for creating a 
Huffman tree based on a frequency table that 
is input together with the embedded coded da- 
ta; and 

Huffman table creation means for creating a 
Huffman table associating a data value in the 
first data with coded data, based on the Huff- 



21 . A data processing apparatus according to Claim 20, 
wherein said first decoded data obtaining 

so means comprises coded data output means for 
translating the embedded coded data into coded 
data based on the association between the coded 
data and the embedded coded data in the transla- 
tion table, and for outputting the coded data, 

55 and said first decoded data obtaining means 

decodes the coded data output from said coded da- 
ta output means based on the Huffman table to ob- 
tain the first decoded data. 
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22. A data processing appa^l^comprising: 
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tentative decoding means for modifying a por- 
tion of embedded coded data encoded by em- 
bedding second data in first data according to 5 
an input parameter, and for tentatively decod- 
ing it based on a coding table to output tentative 
decoded data; 

re-encoding means for re-encoding the tenta- 
tive decoded data to output re-encoded data; 10 
and 

decoding means for determining the parameter 
by comparing the embedded coded data and 
the re-encoded data, outputting tentative de- 
coded data as first decoded data, obtained by ^ 
tentatively decoding, based on the coding ta- 
ble, the embedded coded data having been 
partially modified by said tentative decoding 
means based on the parameter, and for obtain- 
ing second decoded data corresponding to the 20 
parameter. 

23. A data processing apparatus according to Claim 22, 

wherein said tentative decoding means mod- 
ifies a quantization error of coded data including 25 
quantization code and quantization error based on 
the parameter, and tentatively decodes the coded 
data based on a coding table to output tentative de- 
coded data. 

30 

24. A data processing apparatus according to Claim 23, 

wherein said tentative decoding means ro- 
tates a portion of a vector quantization error of cod- 
ed data including vector quantization code and vec- 
tor quantization error based on the parameter, and 35 
tentatively decodes it based on a codebook to out- 
put tentative decoded data. 

25. A data processing apparatus according to Claim 24, 

wherein said tentative decoding means com- *o 
prises: 



the tentative decWBd data, and for detecting a ten- 
tative vector quantization error representing the dif- 
ference between the tentative decoded data and 
the representative vector represented by the vector 
quantization code. 

27. A data processing apparatus according to Claim 26, 

wherein said decoding means comprises 
comparison means for comparing the vector quan- 
tization code with the tentative vector quantization 
code and for comparing the vector quantization er- 
ror and the tentative vector quantization error, 

and when the comparisons by said compari- 
son means result in coincidence, said decoding 
means outputs second decoded data correspond- 
ing to the parameter representing the amount of 
movement by the rotation of the vector quantization 
error, and also outputs the tentative decoded data 
as first decoded data, 

whereas when the comparisons do not result 
in coincidence, said decoding means modifies the 
parameter representing the amount of movement 
by the rotation of the vector quantization error, and 
supplies it to said tentative decoding means. 

28. A data processing apparatus according to Claim 22, 

comprising a plurality of said tentative decod- 
ing means, to which different parameters are re- 
spectively input, 

wherein said decoding means selects one of 
the plurality of said tentative decoding means in ac- 
cordance with the result of the comparison between 
the embedded coded data and the re-encoded da- 
ta, outputting the tentative decoded data output 
from the selected one of said tentative decoding 
means as first decoded data, and also outputting 
second decoded data corresponding to the param- 
eter input to the selected one of said tentative de- 
coding means. 

29. A data processing apparatus comprising: 



inverse vector quantization means for decoding 
the vector quantization code based on the 
codebook to output inverse vector quantization 
data; and 

addition means for adding the rotated vector 
quantization error to the inverse vector quanti- 
zation data output from said inverse vector 
quantization means to obtain the tentative de- so 
coded data. 



encoding means for encoding first data accord- 
ing to a coding rule to output coded data; and 
modification means for modifying the coding 
rule based on second data; 

wherein said encoding means encodes the 
first data according to a coding rule modified by said 
modification means, thereby generating embedded 
coded data in which the second data is embedded. 



26. A data processing apparatus according to Claim 24, 
wherein said re-encoding means comprises 
re-vector quantization means for performing vector 
quantization to detect, based on the codebook, a 
tentative vector quantization code representing a 
representative vector that is most approximate to 



30. A data processing apparatus according to Claim 29, 
wherein said encoding means performs vari- 
able-length encoding according to a coding rule in 
which each data value included in the first data is 
assigned a coded data having a length based on 
the frequency of occurrence of the data value. 
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31 . A data processing apparafWRcording to Claim 30, 

wherein said modification means modifies, 
based on the second data, coded data associated 
with data values to which coded data of the same 
length are assigned. 5 

32. A data processing apparatus according to Claim 31 , 

wherein said modification means modifies, 
based on the second data, coded data associated 
with data values to which coded data of the same 10 
length are assigned in a coding table associating 
each data value with coded data to be assigned to 
the data value based on the frequency of occur- 
rence of each data value in the first data, thereby 
modifying the coding table to create a modified ta- 15 
ble, 

and said encoding means generates embed- 
ded coded data by encoding the first data according 
to the modified table. 

20 

33. A data processing apparatus according to Claim 29, 

wherein said modification means manipu- 
lates, based on the second data, a quantization 
code and a quantization error output by quantizing 
the first data, so that the quantization code and the 25 
quantization error cause a mismatch, 

and said encoding means outputs the manip- 
ulated quantization code and quantization error as 
embedded coded data. 

30 

34. A data processing apparatus according to Claim 29, 

wherein the first data is image data. 
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obtained by variaBPTength decoding the embed- 
ded coded data, thereby restoring the embedded 
coded data into the coded data variable-length en- 
coded according to the entropy coding rule, 

38. A data processing apparatus according to Claim 37, 

wherein said first decoding means comprises 
translation table creation means for creating a 
translation table for translating coded data in a cod- 
ing table for variable-length encoding the first data 
into coded data, based on the frequency of occur- 
rence of each data value in the first data and the 
frequency of occurrence of each data value in de- 
coded data obtained by variable-length decoding 
the embedded coded data. 

39. A data processing apparatus according to Claim 35, 

wherein said first decoding means decodes 
the embedded coded data to obtain tentative coded 
data, 

said second decoding means decodes the 
tentative coded data to output tentative decoded 
data, 

said data processing apparatus further com- 
prising determination means for determining wheth- 
er the tentative decoded data is correct. 

40. A data processing apparatus according to Claim 39, 

wherein said determination means deter- 
mines whether the tentative decoded data is correct 
by encoding the tentative decoded data according 
to the entropy coding rule. 



35. A data processing apparatus comprising: 

first decoding means for decoding embedded 
coded data obtained by embedding second da- 
ta in first data, to obtain coded data encoded 
according to an entropy coding rule and to ob- 
tain the second data; and 
second decoding means for decoding the cod- 
ed data to obtain the first data, 

36. A data processing apparatus according to Claim 35, 

wherein said first decoding means decodes 
the embedded coded data to obtain coded data var- 
iable-length encoded according to an entropy cod- 
ing rule in which coded data of a length based on 
the frequency of occurrence of each data value in 
the first data is assigned. 

37. A data processing apparatus according to Claim 36, 

wherein said first decoding means modifies 
coded data associated with each data value in the 
first data, to which coded data of the same length 
is assigned, based on the frequency of occurrence 
of each data value in the first data and thef requency 
of occurrence of each data value in decoded data 



41 . A data processing apparatus according to Claim 40, 
35 wherein said determination means deter- 
mines whether the tentative decoded data is correct 
by comparing re-encoded data obtained by encod- 
ing the tentative decoded data according to the en- 
tropy coding rule with the tentative coded data. 

40 

42. A data processing apparatus according to Claim 39, 

wherein said first decoding means decodes 
the embedded coded data obtained by manipulat- 
ing, based on the second data, a quantization error 
45 of coded data including quantization code and 
quantization error obtained by quantizing the first 
data, by manipulating the quantization error, there- 
by obtaining the tentative coded data. 

50 43. A data processing apparatus according to Claim 42, 
wherein said determination means deter- 
mines whether a quantization code and a quantiza- 
tion error constituting the tentative coded data are 
correct by comparing the quantization code and the 

55 quantization error constituting the tentative coded 
data with a re-quantization code and a re-quantiza- 
tion error obtained by re-quantizing tentative decod- 
ed data obtained by decoding the quantization code 
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44. A data processing apparatus according to Claim 35, 

wherein the first data is image data. 

5 

45. A data processing system comprising an encoding 
apparatus comprising: 

encoding means for encoding first data accord- 
ing to a coding rule to output coded data; and 10 
modification means for modifying the coding 
rule based on second data; 

wherein said encoding means encodes the 
first data according to a coding rule modified by said « 
modification means to generate embedded data in 
which the second data is embedded, 

said data processing system also comprising 
a decoding apparatus comprising: 

20 

first decoding means for decoding embedded 
coded data to obtain the coded data encoded 
according to the coding rule and to obtain the 
second data; and 

second decoding means for decoding the cod- 25 
ed data to obtain the first data. 

46. A data processing method, 

wherein first data is encoded to output coded 
data, 30 

and a portion of the coded data is modified 
based on the second data so that the second data 
is embedded in the coded data. 

47. A data processing method, 35 

wherein a coding table for encoding first data 
is created, 

the coding table created is modified based on 
second data to create a modified coding table, 

and the first data is encoded based on the *o 
modified coding table to generate embedded coded 
data in which the second data is embedded. 

48. A data processing method, 

wherein embedded coded data encoded by 45 
embedding second data in first data is tentatively 
decoded based on a coding table to output tentative 
decoded data, 

a tentative coding table is created based on 
the tentative decoded data, so 

the embedded coded data is decoded based 
on the coding table and the tentative coding table 
to obtain first decoded data, 

and the coding table is compared with the ten- 
tative coding table to obtain second decoded data, ss 

49. A data processing method, 

wherein a portion of embedded coded data 



encoded by emBllHing second data in first data is 
modified according to an input parameter, and ten- 
tatively decoded based on a coding table to output 
tentative decoded data, 

the tentative decoded data is re-encoded to 
output re-encoded data, 

and the parameter is determined by compar- 
ing the embedded coded data and the re-encoded 
data, and tentative decoded data obtained by ten- 
tatively decoding, based on the coding table, the 
embedded coded data having been partially modi- 
fied based on the parameter, is output as first de- 
coded data, and for second decoded data corre- 
sponding to the parameter is also obtained. 

50. A data processing method, 

wherein first data is encoded according to a 
coding rule to output coded data, 

the coding rule is modified based on second 

data, 

and the first data is encoded according to the 
modified coding rule to generate embedded coded 
data in which the second data is embedded. 

51. A data processing method, 

wherein embedded coded data obtained by 
embedding second data in first data is decoded to 
obtain coded data encoded according to an entropy 
coding rule and to obtain the second data, 

and the coded data is decoded to obtain the 
first data. 

52. A storage medium storing a program, 

wherein first data is encoded to output coded 

data, 

and a portion of the coded data is modified 
based on the second data so that the second data 
is embedded in the coded data. 

53. A storage medium storing a program, 

wherein a coding table for encoding first data 
is created, 

the coding table created is modified based on 
second data to create a modified coding table, 

and the first data is encoded based on the 
modified coding table to generate embedded coded 
data in which the second data is embedded. 

54. A storage medium storing a program, 

wherein embedded coded data encoded by 
embedding second data in first data is tentatively 
decoded based on a coding table to output tentative 
decoded data, 

a tentative coding table is created based on 
the tentative decoded data, 

the embedded coded data is decoded based 
on the coding table and the tentative coding table 
to obtain first decoded data, 



25 



49 EP 1 331 820 A1 



and the coding tabWPcompared with the ten- 
tative coding table to obtain second decoded data. 



55. A storage medium storing a program, 

wherein a portion of embedded coded data 5 
encoded by embedding second data in first data is 
modified according to an input parameter, and ten- 
tatively decoded based on a coding table to output 
tentative decoded data, 

the tentative decoded data is re-encoded to 10 
output re-encoded data. 

and the parameter is determined by compar- 
ing the embedded coded data and the re-encoded 
data, and tentative decoded data obtained by ten- 
tatively decoding, based on the coding table, the 15 
embedded coded data having been partially modi- 
fied based on the parameter, is output as first de- 
coded data, and for second decoded data corre- 
sponding to the parameter is also obtained. 

20 

56. A storage medium storing a program, 

wherein first data is encoded according to a 
coding rule to output coded data, 

the coding rule is modified based on second 
data, 25 

and the first data is encoded according to the 
modified coding rule to generate embedded coded 
data in which the second data is embedded. 



57. A storage medium storing a program, 30 
wherein embedded coded data obtained by 
embedding second data in first data is decoded to 
obtain coded data encoded according to an entropy 
coding rule and to obtain the second data, 

and the coded data is decoded to obtain the 35 
first data. 
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FIG. 4 
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FIG. 8 
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FIG. 10 
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FIG. 11 
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